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SUMMARY 


This  report  examines  and  details  the  activities  of  a  major  U.S.  airline  during  the  period 
of  late  1993  to  late  1997,  as  it  acquired  two  fleets  of  advanced  technology  aircraft,  the 
Boeing  757  and  the  737-500.  The  host  airline  had  planned  to  purchase  767s  during 
the  period  of  the  study,  but  delivery  was  delayed  for  economic  reasons.  The  767  and 
757  is  considered  a  single  fleet  due  to  the  commonality  of  their  cockpits. 

All  three  aircraft  were  equipped  with  electronic  flight  instrument  systems  (EFIS), 
colloquially  known  as  "glass  cockpits."  There  are  aircraft  with  flight  management 
systems  (FMSs),  but  with  traditional  instrumentation  (e.g.  some  models  of  the  B-737- 
300).  But  generally  the  glass  aircraft  have  both  FMSs  and  instrument  panels  that  are 
driven  by  computer-based  color  graphics.  These  are  not  simply  electronic  replications 
of  traditional  aircraft  instruments,  but  are  highly  versatile  displays  that  can  do  what 
traditional  instruments  cannot  (e.g.  the  HSI  moving  map  display,  the  display  of  radar 
returns  on  the  map,  the  display  of  the  wind  vector,  and  the  position  predictor  vector.) 
Prior  to  the  delivery  of  the  first  737-500  in  January  1 994,  the  airline  had  no  glass 
airplanes.  The  most  modern  aircraft  was  the  737-300  non-EFIS  ("round  dial"),  with  a 
modern  FMS  (see  above). 

Although  the  primary  focus  of  the  study  was  upon  flight  training,  we  examined  as  well 
the  technical  support  and  management  of  the  pilots  in  these  fleets,  in  some  cases  very 
detailed  matters,  such  as  checklist  and  procedure  design. 

Questionnaire  data  were  collected  in  three  phases 

Phase  1  -  the  first  day  of  transition  training 

Phase  2  -  approximately  3-4  months  after  transition  training 

Phase  3  -  approximately  12-14  months  after  initial  operating  experience  (IOE) 

A  total  of  150  pilots  who  were  entering  757  transition  training  volunteered  for  the  study. 
Three  were  dropped  during  data  analysis  for  the  first  stage  due  to  incomplete  data 
records.  Of  the  remaining  147,  102  returned  data  forms  in  Phase  II  of  the  study,  and  of 
these  99  pilot  volunteers  also  completed  the  forms  in  the  third  phase. 

Face-to-face  and  telephone  interviews  were  conducted  with  a  sample  of  20  line  pilots, 
as  well  with  flight  instructors,  check  airmen,  management  pilots,  and  ground  school 
instructors 

As  a  side  activity,  at  the  request  of  the  company  a  sample  of  volunteers  going  through 
transition  to  the  737-300/500  was  selected  and  given  the  questionnaires,  before  and 
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after  a  change  in  the  training  program.  The  company  wished  to  have  an  independent 
assessment  of  the  effect  of  the  change.  This  study  will  be  reported  in  a  subsequent 
publication  authored  by  Rebecca  Chute. 

The  757  study  found  that  by  and  large,  pilots  transitioning  to  the  B-757,  most  of  whom 
were  going  to  their  first  glass  cockpit,  had  high  morale,  low  levels  of  apprehension 
about  the  transition,  and  a  generally  positive  attitude  toward  their  training,  and  toward 
cockpit  automation.  They  also  shared  some  concerns,  such  as  what  they  perceived  as 
a  potential  for  a  loss  of  basic  airmanship  skills,  and  an  apprehension  about  having 
sufficient  time  for  extra-cockpit  scanning  (“head  outside").  These  concerns  will  be 
addressed  in  this  report. 

Start-up  Transients 

The  program  was  hampered  in  the  beginning  by  schedule  problems  due  to  uncertain 
aircraft  deliveries,  at  times  resulting  in  insufficient  aircraft  lines  for  the  number  of  pilots 
in  training.  This  in  turn  resulted  in  pilots  having  to  return  to  their  previous  aircraft,  or 
other  aircraft,  before  they  could  later  be  assigned  to  the  757.  At  times  the  opposite 
occurred  --  rapid  acquisition  of  aircraft  resulted  in  pilot  shortages  and  an  acceleration 
of  the  training  schedule.  There  was  also  an  unexpected  bid  off  of  the  757  due  to  what 
pilots  considered  undesirable  flying  schedules,  and  their  disappointment  over  the 
cancellation  of  the  767  order.  The  767’s  ETOPS  capabilities  and  the  promise  of  trans- 
Atlantic  flying  had  been  a  great  motivator  for  bidding  the  757-767  transition. 

The  Continental  program  differed  from  other  programs  in  many  ways,  as  discussed  in 
the  body  of  this  report.  One  significant  difference  was  that  preparation  for  ETOPS 
operations  and  international  flight  were  built  right  into  the  training  syllabus.  All  757 
pilots  emerged  with  ETOPS  LOFT  experience.  First  officers  were  type  rated  in  the  757- 
767.  The  757  was  pressed  into  ETOPS  service  (with  180  minute  certification)  very 
soon  after  the  program  began.  Service  began  with  flights  from  Newark  to  Manchester, 
England,  and  later  Newark  to  Lisbon.  In  spite  of  doubts  about  the  marketing  issues 
raised  by  a  single-aisle  aircraft  in  trans-Atlantic  flight,  the  757  was  an  immediate 
success,  both  with  respect  to  marketing  and  flight.  Ironically,  the  757  flights  to  Europe 
were  so  successful  that  they  were  taken  off  the  route  and  replaced  by  DC-1  Os. 

Pilots  entered  the  transition  program  with  a  far  more  positive  attitude  toward 
automation,  and  less  apprehension  about  being  able  to  make  it  through  the  program, 
than  we  saw  in  previous  field  studies.  We  believe  that  this  is  due  in  part  to  the  fact  that 
advanced  automation,  by  the  time  of  this  study,  no  longer  evoked  emotions  of 
uncertainty,  and,  with  certain  reservations,  had  proven  itself  to  a  skeptical  pilot 
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population.  Other  factors  were  the  generally  positive  attitudes  prevailing  in  an  airline 
struggling  to  emerge  from  a  stormy  recent  past,  marked  by  bankruptcies,  strikes, 
extremely  rapid  expansion  through  mergers  and  acquisitions,  and  mistrust  between 
management  and  labor.  By  the  time  the  first  757  arrived  on  the  property,  there  was  a 
“can  do”  spirit  prevailing  throughout  the  pilot  ranks,  and  the  rest  of  the  airline.  This 
spirit  grew  steadily  during  the  years  of  the  study,  as  we  have  noted  elsewhere  in  this 
report.  Finally,  much  of  the  positive  attitude  can  be  credited  to  the  respect  for  and 
popularity  of  the  two  fleet  managers  of  the  757/767  program,  and  the  fact  that  they 
were  given  a  free  hand  to  pick  the  initial  cadre  of  training  pilots. 

The  program  made  ample  use  of  an  advanced  flight  training  device  (FTD),  computer 
based  training  (CBT),  and  a  full-flight  simulator  (FFS),  modifying  the  training  syllabus 
as  they  went.  Fine-tuning  of  the  training  and  the  use  of  the  devices  took  place  as  the 
program  progressed.  The  program  had  the  usual  start-up  transients.  The  loss  of  the 
767  order  was  a  severe  blow. 

A  Clean  Sheet  of  Paper 

The  terms  “clean  sheet  of  paper”  and  “free  hand"  emerged  time  and  again  to  describe 
the  extent  to  which  the  success  of  the  program  was  the  result  of  unswerving  support 
from  higher  management.  These  phrases  represented  not  only  the  all-important 
perception  of  support,  but  the  practicalities  -  that  fleet  manager’s  requests  were  taken 
seriously,  and  that  management  did  not  quibble  or  “nickel-dime”  the  managers  of  the 
training  program.  The  fact  that  this  support  seemed  unusual,  and  needed  to  be 
commented  on,  leads  the  authors  to  believe  that  the  lot  of  a  fleet  manager  had  not 
always  been  a  happy  one.  More  will  be  said  of  this  in  chapters  VIII  and  IX. 
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I.  BACKGROUND  TO  THE  STUDY 


A.  CONTINENTAL  AIRLINES  -  SUMMER  OF  1 993 


In  August  1993,  flight  management  from  Continental  Airlines  (CAL),  based  in  Houston, 
approached  the  first  author  and  asked  him  to  consult  with  them  on  transition  training  for 
pilots  who  would  be  transferring  to  the  company's  two  new  fleets  of  aircraft,  the  Boeing 
757-767  and  the  Boeing  737-500.  Two  weeks  later  in  Los  Angeles,  accompanied  by 
Dr.  Everett  Palmer  of  NASA  Ames  Research  Center,  he  made  a  presentation  and 
proposal  to  company  officials,  resulting  in  this  study,  a  cooperative  project  between 
NASA,  Continental,  and  the  University  of  Miami.  Captain  David  Lynn,  fleet  manager  for 
the  757-767  program,  was  named  to  be  Continental’s  point  of  contact  for  the  study.  In 
1996  Captain  Lynn  took  over  the  737  program,  and  was  replaced  by  Captain  David 
Sanctuary,  who  remained  our  point  of  contact  to  the  end  of  the  study. 

The  fleet  at  the  time  consisted  of  the  DC-9,  MD-80,  DC-10,  A-300-2B  (3-pilot, 
traditional  instrumentation,  not  to  be  confused  with  the  A-300-600,  a  2-pilot  EFIS 
aircraft),  B-727,  B-737-1 00/200/300  (non-EFIS1),  and  B-747-1 00/200.  In  addition 
Continental  operated  Continental  Express,  flying  ATR-42,  ATR-72,  EMB-120,  and 
Beech  1900.  The  737  was  the  largest  fleet,  and  will  remain  so.  The  most  advanced 
cockpit  in  the  fleet  was  the  B-737-300,  with  FMC  but  no  EFIS,  so  CAL  had  no 
experience  with  EFIS  (“glass  cockpit”)  at  that  time.  The  fleet  at  Continental  proper 
numbered  about  460  [note:  henceforth  we  shall  consider  only  Continental  Airlines,  and 
ignore  Continental  Express  and  Continental  Connection],  The  A-300  fleet  has  since 
been  retired,  and  some  B-727s  have  been  retired.  Older  models  of  the  B-737  are  also 
being  retired  as  the  500  models,  as  well  as  the  “next  generation”  Boeing  737s,  are 
added  to  the  fleet. 

Early  in  the  next  century  Continental  will  have  an  all-glass,  all-Boeing  fleet.  In  1999 
alone  Continental  will  take  delivery  of  58  new  Boeing  aircraft.  The  consolidation 
around  Boeing  aircraft  will  result  in  CAL’s  fleet  having  five,  rather  than  the  present  nine 
major  model  types,  with  predicted  savings  of  $  50  million  per  year  (Proctor,  1998a). 


1  A  glossary  of  terms,  mainly  those  dealing  with  flight-deck  automation,  can  be  found  in 
Appendix  B. 
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Under  the  leadership  of  CEO  Gordon  Bethune,  Continental  will  move  in  a  very  short 
time  from  an  essentially  obsolete  fleet  to  one  of  the  youngest  in  the  industry. 


B.  A  BRIEF  HISTORY  OF  CONTINENTAL 


Since  this  study  was  concentrated  at  one  airline,  and  one  with  a  turbulent  financial  and 
labor  history  through  the  ‘80s  and  into  this  decade,  it  is  necessary  to  understand  some 
of  the  history  of  the  company.  The  authors  do  not  take  sides  in  this  discussion,  but  try 
to  present  an  dispassionate  discussion  and  understanding  of  how  the  company’s 
background  and  culture  developed,  and  how  it  impacted  the  present  study. 

Continental,  as  we  know  it  today,  is  the  product  of  many  tributaries,  including  the 
original  Continental  (“Old  Continental”  as  it  is  called  by  pilots),  Pioneer,  Texas 
International,  Frontier,  New  York  Airways,  and  People  Express.  Continental  was 
founded  in  1937  out  of  a  Southern  California  company,  Varney  Speed  Lines.  The 
following  year  Robert  F.  Six  became  president,  and  led  the  company  for  over  40  years. 
He  built  a  California-based  company  concentrating  on  providing  passenger  service  to 
the  southwest  and  later  Hawaii.  In  1953  Continental  acquired  Pioneer  Airlines,  with  16 
destinations  in  the  west.  Six  moved  the  company  from  El  Paso  to  Denver,  and  in  1963 
established  the  headquarters  in  Los  Angeles.  In  1968  Continental  formed  a  subsidiary, 
Air  Micronesia,  to  serve  the  islands  of  the  Pacific.  By  1980  Continental  had  180 
aircraft.  Today  it  has  earned  its  position  in  the  “middle  three"  of  U.S.  air  carriers 
(Northwest,  Continental,  and  USAirways),  and  is  currently  engaged  in  forging  an 
alliance  with  Northwest. 

The  Airline  Deregulation  Act  of  1978  ushered  in  a  period  of  extreme  turbulence  in  the 
airline  industry.  The  experience  at  Continental  was  more  than  turbulent.  These  were 
the  years  of  “merger  mania.”  In  1981,  the  original  Continental  was  purchased  by 
Frank  Lorenzo’s  holding  company,  Texas  Air.  Continental  was  later  merged  with  Texas 
International  Airline  (formerly  Trans-Texas),  but  kept  the  Continental  name.  It  was 
called  by  many  “New  Continental.”  The  following  year  Robert  Six,  at  the  age  of  74, 
retired  from  the  airline. 

Lorenzo’s  Texas  Air  Corporation  (TAC)  which  already  owned  New  York  Airways, 
bought  Continental,  then  acquired  Frontier,  New  York  Airways,  and  People  Express. 

In  1986  Texas  Air  Corporation  purchased  Eastern  Airlines.  In  January  1987  TAC 
folded  New  York  Air,  Frontier,  and  People  Express  into  Continental,  resulting  in  two 
companies  of  about  the  same  size,  Continental  and  Eastern. 

In  1 983  Lorenzo  took  Continental  into  bankruptcy.  The  conventional  wisdom  was  that 
there  was  no  financial  justification  for  his  move,  that  he  was  using  the  bankruptcy  laws 
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to  defeat  the  unions.  Lorenzo  allegedly  took  advantage  of  the  bankruptcy  laws  to 
abrogate  labor  contracts  and  impose  lower  wages  and  longer  hours.  In  the  case  of 
pilots,  this  meant  more  flying  time  for  less  pay.  The  Air  Line  Pilots  Association  (ALPA) 
joined  the  machinists  on  strike.  Later  Congress  plugged  that  loophole  in  U  S. 
bankruptcy  law. 

The  strike  was  not  well  disciplined.  Many  pilots  crossed  the  picket  lines  after  a  brief 
gesture  of  supporting  the  strike.  The  strike  ended  in  1985,  as  the  company  emerged 
from  bankruptcy.  ALPA  was  decertified  as  Continental’s  bargaining  agent  Later,  in  a 
certification  election,  Continental  pilots  voted  to  create  their  own  union,  the 
Independent  Association  of  Continental  Pilots  (IACP).  This  move  has  interesting 
implications  for  this  study,  and  they  will  be  discussed  later. 

In  1990,  Lorenzo  was  forced  to  sell  his  holdings  in  Continental  and  relinquish  control. 
Immediately  following  this,  management  again  took  the  company  into  bankruptcy.  In 
1993  Continental  again  emerged  from  bankruptcy,  and  began  the  process  of 
rehabilitating  the  airline. 

By  1993,  when  this  study  began,  there  was  a  spirit  of  rebuilding,  and  a  cautious 
optimism  at  Continental.  A  large  fleet  of  glass  aircraft  was  ordered  from  Boeing  (737- 
500,  757,  767,  and  later  the  777),  and  they  became  not  only  the  backbone  of  a  new 
and  modernized  fleet,  but  also  a  symbol  of  optimism  and  hope  for  the  airline.  The  first 
737-500  arrived  in  January  1994,  and  the  first  of  an  initial  order  of  757s  in  May  1994. 
The  767  order  was  delayed,  then  canceled,  and  later  reinstated.  The  impact  of  the  off- 
and-on  767  order  on  the  crews,  the  transition  program,  and  this  study,  will  be  discussed 
later.  In  1997  Continental  and  Boeing  signed  an  agreement  for  Continental  to  become 
one  of  several  “all  Boeing”  airlines.  It  placed  a  large  order  for  the  737-700,  and  later 
became  the  domestic  launch  customer  for  the  737-800.  The  767  order  was  reinstated. 
The  first  777  delivery  took  place  in  September  1998.  During  the  year  of  this  writing 
(1998),  Continental  acquired  64  new  Boeing  transports. 

Continental,  along  with  most  of  the  larger  airlines  in  the  U.S.,  was  enjoying  a  period  of 
prosperity  and  profits,  high  load  factors,  fleet  modernization,  and  an  expanding  route 
structure  (Shifrin,  1998).  In  1996  and  1997  Continental  was  cited  time  and  again  by 
business  publications  and  polls  as  one  of  the  leading  examples  of  a  “turn  around” 
company,  both  in  its  financial  success  and  the  quality  of  its  passenger  service.  Much  of 
the  credit  has  been  attributed  to  the  leadership  style  of  Gordon  Bethune  (Bethune  and 
Huler,  1998). 
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C.  THE  HIGH  TECHNOLOGY  COCKPIT 


The  last  two  decades  have  witnessed  the  rapid  and  wide-spread  development  of  an 
entirely  new  cockpit  technology,  based  on  the  capabilities  of  the  microprocessor  and 
color  graphics.  We  will  not  attempt  to  review  the  literature,  history,  or  development  of 
cockpit  automation,  as  it  is  well  reviewed  elsewhere  (see  Sarter  and  Woods,  1994; 
Woods  and  Sarter,  1992;  Rudisill,  1994;  Flint,  1995;  Billings,  1997;  Wiener,  1988, 

1989;  Wiener  and  Nagel,  1988;  FAA,  1996).  For  all  of  automation’s  astounding 
capabilities,  doubts  were  expressed  about  the  human  factors  issues  raised  by  robotic 
flight.  For  an  early  version,  see  Wiener  and  Curry  (1980),  and  for  more  recent  writings, 
see  Last  (1997),  Learmont  (1996),  and  Foreman  (1996). 

Would  the  average  pilot  be  able  to  manage  the  automation  and  its  many  modes? 
Would  the  hardware  and  software  be  able  to  live  up  to  its  claims  for  workload 
reduction,  thus  making  it  possible  to  eliminate  the  flight  engineer’s  position,  and  fly 
large  jet  aircraft,  over  oceans,  with  a  two-pilot  crew?  Would  automated  flight  invite 
operator  “blunders”  (large  errors)  as  seen  in  other  applications  of  automation.  Would 
pilots  “fall  out  of  the  loop”  and  not  be  able  to  keep  up  with  the  airplane?  Would  manual 
flying  skills  become  degraded  (see  Figure  1-1 )?  Some  of  the  doubts  harbored  by  pilots 
can  be  seen  in  the  following  report  to  the  NASA  Aviation  Safety  Reporting  System 
(ASRS). 

[Note:  throughout  this  report  we  include,  for  illustrative  purposes,  reports  selected  from 
a  search  of  the  NASA  ASRS  database  for  cases  dealing  both  with  automation  and 
training.  The  ASRS  form  does  not  ask  for  information  about  the  employer  of  the 
reporter,  or  identify  in  any  form  the  carher(s)  involved  in  the  report.  If  the  carrier  were 
identified  by  the  reporter,  the  information  would  not  be  stored  in  the  database. 
Accordingly,  the  reports  which  are  sampled  and  included  in  the  text  and  in  Appendix  H 
are  probably  not  from  Continental  Airlines,  and  they  may  or  may  not  concern  B-757 
aircraft.  We  chose  our  cases  sthctly  for  their  subject  matter  interest,  and  they  should 
not  be  thought  of  as  reports  concerning  Continental  crews,  or  even  necessarily  B-757 
aircraft]. 

NARRATIVE:  During  IOE  training  en  route  PHL  to  CLE  was  given  clearance  to  cross 
10  miles  east  of  YNG  VORTAC  at  24,000'.  In  discussion  with  check  airman  on  best 
method  to  enter  this  info  into  FMC,  I  decided  to  start  down  and  then  work  on  FMC  in 
descent.  I  inadvertently  selected  10,000'  into  flight  guidance  system.  Again  we  went 
heads  down  to  concentrate  on  programming  FMC  for  descent  path.  Moments  later  CLE 
center  requested  our  altitude.  We  looked  up  as  we  were  through  22,000',  leveled  out  at 
21,000'.  We  informed  Center.  Weather  was  clear  and  controller  just  said  to  maintain 
21,000',  apparently  there  was  no  conflicting  traffic.  This  is  not  a  new  problem 
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Automation  has  taken  over  in  the  cockpit.  Computers  are  not  learned  overnight  and  need 
hands  on  operating  experience  It  all  comes  back  to  "fly  the  airplane  first!" 

ACCESSION  NUMBER:  116912 


Early  Studies 

As  early  as  1977  the  first  alarm  was  sounded  by  the  late  Elwyn  Edwards  (1977),  who 
examined  for  the  first  time  the  broad  question  of  human  factors  of  cockpit  automation. 

At  the  same  time  concerns  were  being  expressed  in  the  U.S.  Congress.  Two 
Congressional  reports  identified  automation  as  a  safety  problem  for  the  coming  decade 
(U.S.  House  of  Representatives,  1977;  U.S.  Senate,  1980).  There  was  much  talk  of 
“the  automation  problem,”  but  no  person  nor  any  agency  was  prepared  to  say  with  any 
certainty  what  “the  problem”  was.  In  1979  NASA  Ames  Research  Center  was  tasked 
with  examining  the  safety  implications  and  human  factors  in  automated  flight.  The 
Congressional  subcommittees  had  no  trouble  recognizing  the  positive  side;  what  they 
wanted  to  know  was  whether  there  was  a  “down  side.”  Quite  simply,  were  there  also 
adverse  consequences  of  the  new  flight  decks  that  the  manufacturers,  regulators,  and 
future  operators  were  overlooking? 

The  project  was  assigned  to  Dr.  Renwick  Curry,  then  of  NASA  Ames,  and  Professor 
Earl  Wiener,  on  leave  at  Ames  from  the  University  of  Miami.  Their  collaboration 
produced  a  comprehensive  report  (Wiener  and  Curry,  1980)  on  the  human  factors  of 
cockpit  automation,  proceeding  beyond  Edwards’  initial  work  (1977).  They  produced  a 
list  of  15  guidelines  for  the  design  and  utilization  of  cockpit  automation.  Guidelines 
from  other  authors  followed  (see  Billings,  1997).  Following  the  publication  of  their  1980 
paper,  Wiener  and  Curry  conducted  three  field  studies  of  the  adaptation  of  the  new 
aircraft  into  the  fleets  of  several  airlines  (Curry,  1985;  Wiener,  1985b,  1989). 

The  Advance  of  the  Glass  Cockpit 

The  decade  of  the  1980’s  saw  the  appearance  of  the  new,  electronically  sophisticated 
transport  aircraft.  The  Boeing  767  was  followed  shortly  by  the  757,  and  later  glass 
derivatives  of  the  737  and  the  MD-80.  In  the  1990's  a  family  of  original  aircraft  were 
produced  by  Airbus  Industrie:  the  A-3 19-320-321,  the  A-310,  A-330,  A-340,  and  the 
derivative  A-300-600.  The  A-320  series  took  automation  to  a  higher  level  than  the  first 
generation  of  glass  aircraft,  typified  by  the  757-767  and  the  A-310,  introducing  fly-by- 
wire  with  the  side-arm  controller  and  other  advanced  capabilities.  Douglas  fielded  the 
derivative  MD-1 1 ,  Fokker  produced  the  F-1 00,  and  new  models  of  Boeing’s  best-selling 
737  soon  appeared.  The  long-haul  market  today  is  dominated  by  a  glass  derivative  of 
the  traditional  747,  the  747-400,  but  the  smaller  A-340  and  the  B-777  show  promise  of 
being  the  dominant  long-haul  aircraft  of  the  next  two  decades  (Proctor,  1988). 
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The  new  FMS  and  glass  aircraft  were  considered  a  great  success.  The  decision  of  the 
President’s  Task  Force  on  Aircraft  Crew  Complement  (McLucus,  Drinkwater,  and  Leaf, 
1981 )  to  allow  two-pilot  operation  of  the  new  jets  proved  to  be  wise.  This,  coupled  with 
up  to  180  minute  ETOPS  authority,  brought  a  new  era  of  economical  trans-oceanic 


FIGURE  1-1.  Concern  over  potential  for  skill 
loss,  third  phase  of  experiment. 

operations  for  two-engine,  (generally)  two-pilot  glass  cockpit  aircraft.  Under  U.  S. 
federal  aviation  regulations  [FARs],  for  two-engine  aircraft,  three-pilot  crews  are 
required  for  flights  over  eight  hours,  and  four  pilots  when  the  flight  is  scheduled  for  over 
12  hours.  The  success  of  ETOPS  operations  was  summarized  in  a  news  item. 


ETOPS  RECORD.  Boeing  767  transports  have  logged  more  than  1  million  ETOPS 
flights  with  57  airlines.  According  to  Boeing  statisticians,  767  operators  now  log  more 
than  13,000  ETOPS  flights  a  month,  many  of  them  across  the  North  Atlantic..  Through 
May  [1998],  Boeing-built  twin-engine  transports  had  accumulated  more  than  1.2  million 
ETOPS  flights,  according  to  the  manufacturer.  Proctor  (1998b). 


Doubts  and  Reservations 


Still  there  were  nagging  doubts  about  human  factors.  Pilots  were  evenly  divided  on  the 
workload  issue;  many  interviewed  by  the  authors  remarked  “I’ve  never  been  so  busy  in 
my  life”  [flying  the  advanced  cockpit].  We  heard  this  comment  over  and  over.  There 
was  genuine  concern  over  not  only  workload,  but  potential  for  skill  degradation  (“loss  of 
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scan”  as  the  pilots  call  it),  though  to  this  day  there  have  been  little  data  put  forth  to 
support  claims  of  skill  loss.  In  the  single  study  that  we  are  aware  of,  Veillette  (1995) 
demonstrated  a  significant  loss  of  manual  skills  in  crews  flying  the  automated  cockpit. 
Veillette’s  work  is  a  worthy  beginning,  but  more  investigation  of  this  issue  is  needed, 
especially  as  longer  and  longer  flights  are  anticipated.  With  8000-mile  legs,  and 
augmented  crews,  one  can  easily  imagine  flight  schedules  in  which  pilots  will  make  as 
few  as  one  takeoff  and  landing  per  month.  If  their  captain  is  up  for  a  checkride  in  the 
near  future,  it  could  be  even  less  for  the  other  crew  members.  Figure  1-1  above 
displays  the  responses  to  the  probe  on  skill  loss.  To  see  the  responses  during  all  three 
phases  of  the  study,  see  page  A-2. 

Two  comments  that  we  received  in  our  open-ended  questionnaire  items  (Chapter  VI) 
were: 


[Flying  glass  results  in]  more  management  and  less  hands-on  Because  of  the  automation 
being  almost  fool  proof,  I  tend  to  hand  fly  to  15,000  more  often  The  systems  work  well, 
but  I  need  to  keep  basic  flying  skills  in  tune. 

I  bid  off  the  757  because  of  the  automation  and  bad  trips.  The  only  thing  I  really  miss 
about  the  757  automation  is  the  printer.  The  old  technology  is  real  flying,  and  it’s  fun. 
The  old  technology  makes  you  a  better  pilot,  by  hand  flying  and  using  your  brain 


The  concern  over  skill  loss  and  a  variety  of  other  factors  on  the  part  of  one  pilot  can  be 
seen  in  the  following  ASRS  report. 

NARRATIVE:  Descent  from  FL200  to  12000',  using  FMC  navigation  and  autopilot 
Approx  1 5000'  entered  tops,  encountered  moderate  to  severe  turbulence,  heavy  rain. 
Almost  simultaneously  ATC  cleared  to  cross  40  southwest  LRP,  at  12000'.  LRP  not 
available  immediately  due  not  auto  select  on  VOR,  off  screen  on  CRT  Captain  (PNF) 
scrambled  to  find  the  runway  chart  to  get  the  VOR  frequency  while  I  got  engine  anti-ice 
and  ignition  turned  on.  Then  Captain  began  adjusting  radar  to  find  out  why  we  were 
getting  heavy  rain  and  turbulence.  When  DME  finally  locked  on  LRP,  it  read  3 1  nm  (SW 
of  LRP).  I  deployed  spoilers  and  turned  off  auto  thrust.  Rain  and  turbulence  worsened  in 
descent.  As  we  approached  12000',  I  observed  airspeed  decreasing.  Not  immediately 
realizing,  due  to  concern  about  the  extreme  turbulence,  that  the  autopilot  was  leveling  the 
aircraft  at  1 2000'  w/o  auto  thrust  available,  I  disconnected  the  autopilot.  The  aircraft  was 
trimmed  nose  down  and  continued  descent  below  12000'.  The  captain  recognized  the 
problem  immediately  and  called  out,  "altitude."  Flew  the  aircraft  back  to  12000'  and  re¬ 
engaged  autopilot.  Minimum  altitude  approx  11800'.  Contributing  factors:  proficiency— I 
am  junior  on  a  wide  body,  have  been  mostly  assigned  for  last  6  months  as  relief  pilot 
(cruise  only)  or  with  restricted  Captain  Consequently,  flew  1  leg  in  Oct,  2  in  Nov  ,  1  in 
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Dec.,  none  in  Jan,  1  in  Feb.,  none  in  Mar  This  was  only  my  sixth  leg  in  6  months.  Crew 
shortage~I  am  in  the  middle  of  widebody  transition  training  (completed  FAA  oral  3/29. 
Released  from  training  due  to  a  backed  up  simulator  schedule).  Due  to  crew  shortage, 
sent  out  on  wide  body  trip  during  transition  training.  ATC  procedures— assignment  of  a 
crossing  restriction  only  10  nm  from  the  crossing  fix,  using  a  navaid  which  is  behind  an 
aircraft  using  FMC  equip,  imposed  an  excessive  workload  on  the  crew  with  too  little  time 
to  set  it  up.  Fatigue— I  was  extremely  fatigued  after  being  unable  to  sleep  in  the  hotel  in 
Paris.  Hotel  is  noisy  during  the  day  when  crew  is  sleepy,  stuffy  at  night.  Company  refuses 
to  change  hotel.  (I  do  not  smoke  or  drink  alcohol.) 

Recommendations:  The  issue  of  proficiency  of  relief  pilots  on  long  range  flights  should  be 
addressed.  Captains  in  the  widebody  operation  on  our  airline  do  not  feel  obligated  to  give 
legs  to  the  relief  pilot.  Once  having  initiated  transition  training  on  new  equip,  a  pilot 
should  not  be  required  to  operate  a  previously  qualified  equipment  type  w/o  at  least  one 
simulator  refresher  period  Constant  crew  shortages  are  destroying  pilot  personal  lives.  I 
am  beginning  to  believe  that  scheduled  airline  pilot  staffing  levels  need  to  be  addressed  by 
the  FAR’s.  This  is  a  complex  subject,  but  our  pilot  group  is  experiencing  intense  turmoil 
over  the  effects  of  crew  shortages.  ATC  should  avoid  short  range  crossing  restrictions. 
Controllers  should  be  trained  on  operational  characteristics  of  FMC  aircraft  (e.g.,  navaids 
behind  the  aircraft  are  not  readily  accessible)  Pilot  working  agreements  do  not  provide 
adequate  leverage  to  ensure  that  pilots  are  given  suitable  hotel  accommodations. 
Unsuitable  hotels  are  second  only  to  crew  shortages  as  the  major  problem  in  flight  ops  on 
our  airline.  Hotels  are  changed  constantly  to  reduce  costs,  and  many  pilots  are 
complaining  about  fatigue  due  to  inadequate  rest  on  layovers.  ACCESSION  NUMBER: 
108752 


Likewise  the  question  of  relative  workload  in  the  automated  cockpit  is  still  open.  For  a 
review,  see  Wiener,  1993a.  A  simulator  study  by  Wiener,  Chidester,  Kanki,  Palmer, 
Curry,  and  Gregorich  (1991)  compared  performance  as  well  as  pilot  opinion  of  crews 
operating  a  DC-9  and  its  glass  derivative,  the  MD-88,  flying  the  same  LOFT  scenario. 
The  perceived  workload  was  greater  for  the  MD-88  pilots.  However,  the  mean 
differences  were  small,  and  this  is  but  one  study  by  which  to  judge  a  very  complicated 
issue.  Is  the  workload  higher  or  lower  in  a  glass  cockpit  or  a  traditional  cockpit?  This 
question  remains  to  be  answered.  Responses  to  two  of  the  attitude  probes  dealing  with 
automation  and  workload  are  shown  in  Figure  1-2  and  1-3  below.  Graphs  showing  data 
for  all  three  phases  are  on  pages  A-13  and  A-8. 

In  a  bit  of  irony,  the  FAA,  citing  concerns  over  the  workload  induced  by  automation  on  a 
short  leg,  stopped  Mesa  Airlines  from  flying  from  Ft.  Worth  to  Houston  ( AWST ,  May  5, 
1997,  p.  15).  This  prompted  a  letter  from  DC-9  pilot  Bernie  Harrigan,  who  comments 
(AWST,  June  30,  1997,  p.  6)  “I  thought  such  equipment  was  designed  to  reduce 
workload.  It  would  take  me  no  more  than  10  seconds  to  ‘program’  my  DC-9  for  such  a 
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1 3c  There  is  too  much  workload  below 
10,000  feet  and  m  the  terminal  area. 
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FIGURE  1-2.  Attitude  toward  workload  in  terminal  areas. 


flight.  How  far  have  we  come,  and  where  are  we  going?”  Pilot  Harrigan  might  be  even 
more  perplexed  at  an  article  in  which  the  purchaser  of  the  EMB-145  said  that  their 
models  “will  not  be  equipped  with  an  FMS  chiefly  because  those  systems  require  too 


FIGURE  1-3.  Attitude  toward  automation  and  workload  overall. 


1-9 


much  head-down  time  in  the  cockpit  and  provides  no  ‘payback’  during  short-haul 
flights”  (Phillips,  1997).  The  industry  had  been  told  that  automation  would  relieve 
cockpit  workload.  The  question  of  excessive  head-down  time  appears  in  all  of  the  field 
studies  and  opinion  polls  that  we  have  mentioned. 

Students  of  glass  cockpit  human  factors  have  also  been  concerned  about  human  error 
rate  and  severity.  Some  (see  Wiener,  1988,  1989;  Woods  and  Sarter,  1995)  have 
hypothesized  that  automated  flight  invites  rare  but  large,  high-consequence  errors 
(“blunders”)  by  the  very  nature  of  digital  systems.  Results  from  the  LOFT  study 
mentioned  above  did  not  support  this  view:  the  error  severity  was  no  different  when 
comparing  crew  errors  committed  in  the  DC-9  and  the  MD-88. 

The  matter  of  mode  errors  appeared  in  the  training  programs  to  be  vexing:  pilots 
transitioning  to  glass  for  the  first  time  had  difficulty  understanding  and  properly  utilizing 
the  autoflight  modes.  “Mode  error”  is  a  broad  term:  it  encompasses  selecting  an 
inappropriate  mode,  not  understanding  the  implications  of  choice  of  mode,  not  realizing 
what  mode  was  engaged,  and  failing  to  recognize  that  a  change  in  mode  had  been 
made  not  by  pilot  selection,  but  by  the  FMS.  Mode  errors  were  to  play  a  vital  role  in  the 
series  of  glass  cockpit  accidents  that  was  to  follow  (Degani,  Shafto,  and  Kirlik,  1995; 
Degani,  Shafto,  and  Kirlik,  in  press;  Hughes  and  Dornheim,  1995;  Hughes,  1995; 
Phillips,  1995). 

Glass  Cockpit  Accidents 

In  June,  1988  an  Air  France  A-320  crashed  while  making  a  low  pass  over  the  field  at  an 
air  show  in  Germany.  Misuse  of  automation  modes  was  blamed.  Less  than  two  years 
later  another  A-320  crashed  in  Bangalore,  India  due  to  mode  mismanagement. 
Following  this  came  a  string  of  accidents  and  dramatic  incidents  involving  first  Airbus, 
then  other  manufacturers’  high-technology  aircraft  (Sekigawa  and  Mecham,  1996). 

With  the  situation  appearing  to  be  somewhat  out  of  hand,  Aviation  Week  and  Space 
Technology  published  a  two-part  series  on  the  automated  cockpit,  edited  by  David 
Hughes  and  Michael  Dornheim  (1995).  We  will  not  comment  further  on  these 
accidents,  as  they  are  well  covered  elsewhere  (Hughes  and  Dornheim,  1995;  Billings, 
1997),  as  well  as  the  official  accident  reports,  most  of  which  have  been  translated  into 
English.  For  a  detailed  discussion  of  human  error  management,  see  Reason  (1990) 
and  Wiener  (1993b).  We  do  not  cover  in  this  study  the  expanding  area  of  the  effect  of 
national  and  regional  culture  on  accident  rates,  acceptance  of  modern  technology  and 
CRM  training.  The  reader  wishing  this  is  directed  to  Johnston  (1993a),  and  to 
Helmreich  and  Merritt  (1998). 

Starting  In  December  1995,  it  was  the  Boeing  757's  turn  to  be  the  center  of  attention. 
First,  an  American  Airlines  B-757  (Flight  965)  crashed  into  a  mountain  while  initiating 
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an  approach  to  Cali,  Colombia  (Aeronautica  Civil  of  the  Republic  of  Colombia,  1996  [in 
English]).  This  was  the  first  hull  loss  accident  involving  a  major,  U.S. -operated  glass 
airplane. 

Two  more  757  accidents  followed  in  short  order.  The  first  was  a  Birgenair  aircraft  that 
crashed  off-shore  near  Puerto  Plata,  Dominican  Republic  (Phillips,  1996b).  In  October 
of  that  year  an  Aeroperu  757  crashed  off-shore  of  Lima  (McKinna,  1996b).  The  Cali 
accident  is  regarded  by  many  as  a  turning  point  in  the  brief  history  of  the  glass  cockpit: 
a  flight  crew  without  a  clear  picture  of  where  they  were  or  a  clear  plan  for  the  approach 
once  they  accepted  a  runway  change,  and  over-relying  on  automation,  when  hand 
flying  and  basic  instruments  would  have  been  sufficient.  The  U.S.  aviation 
establishment  noted  that  heretofore  the  automation-induced  accidents  occurred 
exclusively  on  foreign  soil,  and  were  the  work  of  foreign  carriers,  and  mostly  foreign 
(Airbus)  manufactured.  This  time  it  was  a  U.S.  carrier.  (At  the  time  of  this  writing  there 
has  never  been  a  crash  of  a  large  glass  cockpit  passenger  aircraft  in  the  U.S.  There 
have  been  crashes  of  glass-equipped  commuter  aircraft.) 

Although  there  was  the  inevitable  disagreement  about  the  causes  of  the  individual 
accidents,  and  the  role  that  automation  played,  and  considerable  denial  on  the  part  of 
the  manufacturers  (see  Hughes  and  Dornheim,  1995;  Dornheim,  1995),  more  and  more 
persons  in  the  industry  were  willing  to  admit  that  there  were  serious  problems  at  the 
pilot-automation  interface,  as  predicted  by  Edwards  (1977)  and  Wiener  and  Curry 
(1980).  One  concern  was  the  relatively  weak  role  played  by  the  FAA  certification 
process  in  guaranteeing  safe  designs.  In  defense  of  the  FAA,  it  must  be  recognized 
that  certification  standards  simply  did  not  exist.  The  certification  requirements  of  the 
Federal  Aviation  Regulations  (FARs),  Part  25  were  based  on  an  earlier  era  of 
autoflight,  when  sophisticated  flight  management  systems  (FMSs)  were  unknown.  The 
FAA  could  not  be  expected  to  enforce  what  did  not  exist.  FAA  certification  personnel 
were  well  versed  in  traditional  areas:  propulsion,  aerodynamics,  structures,  and 
guidance.  They  were  not  prepared  for  the  flight  management  systems  of  the  1980's. 

Recognizing  the  need  to  develop  human  factors  certification  standards  for  modern 
autoflight,  the  FAA  appointed  a  committee,  chaired  by  Dr.  Kathy  Abbott  of  NASA- 
Langley,  and  Stephen  Slotte  and  Donald  Stimson  of  the  FAA,  to  study  the  interface 
problem  and  make  recommendations  to  the  FAA  for  implementation  of  certification 
standards.  Their  report  (FAA,  1996)  contains  a  long  list  of  recommendations  that  will 
form  the  blueprint  for  future  design  and  certification  of  pilot-automation  interfaces 
(North,  1998). 

The  FAA  study  in  turn  brought  a  flurry  of  activity  in  the  U.S.  and  in  Europe.  The 
research  community  saw  the  report  as  a  blueprint  for  studies  that  needed  to  be  done  to 
support  the  FAA’s  certification  effort  with  timely  human  factors  data.  In  the  U.  S.,  the 
“alphabet”  organizations  also  wanted  their  influence  to  be  felt.  For  example,  the  Air 
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Transport  Association  (ATA)  appointed  Captain  Frank  Tullo  (Continental)  to  head  their 
Human  Factors  Committee.  The  Automation  Subcommittee  is  chaired  by  Dr.  Tom 
Chidester  (American). 

The  inevitable  question  arising  out  of  the  accidents  is  whether  glass  aircraft  are  more 
or  less  safe  than  traditional  models.  Boeing  produced  data  (Boeing,  1997;  Daily,  1997) 
that  showed  the  mean  time  between  hull  loss  accidents  to  be  considerably  greater  for 
glass  than  for  conventionally  instrumented  aircraft  Confirmation  of  these  results  came 
from  Airbus  (Davis,  1997;  Sparaco,  1998).  An  example  of  a  classic  controlled  flight  into 
terrain  (CFIT)  accident  in  an  old  technology  aircraft  (Boeing  T-43,  the  military  version  of 
the  B-737-200)  which  occurred  in  Bosnia  (Phillips,  1996a). 

The  data  provided  by  these  two  manufacturers  are  difficult  to  interpret,  since  the  old 
technology  planes  flew  more  in  earlier  years,  where  many  things  were  different  --  less 
safety  equipment  apart  from  the  flight  guidance  systems,  ATC  control  and  weather 
information  were  less  developed  than  in  the  recent  18  years  of  glass  cockpit 
operations,  and  the  warning  and  alerting  devices  that  we  know  today  are  fairly  recent. 

In  addition,  the  glass  aircraft  are  superior  in  many  ways  apart  from  instrumentation: 
better  wings,  better  engines,  and  better  cockpit  procedures,  perhaps  even  the  CRM 
movement,  to  mention  only  a  few. 

The  original  Boeing  data  were  computed  before  the  757  accidents.  Later  the  figures 
were  recomputed,  including  the  757  crashes,  and  the  new  technology  aircraft  still  had 
superior  safety  records. 

Perhaps  we  are  asking  the  wrong  question.  We  should  be  focusing  not  on 
comparisons  of  glass  and  conventional  aircraft,  but  on  recognition  of  the  fact  that  all 
new  transport  aircraft  will  soon  be  FMS  equipped,  and  will  probably  be  glass  equipped. 
The  question  should  be  reworded  and  made  more  constructive.  The  question  we 
propose  is  simply  this:  what  can  manufacturers,  operators,  and  government  do  to 
maximize  the  reliability  of  human  and  machine  performance  of  modern  aircraft  and 
enhance  safety?  (See  McKinna,  1997). 

In  this  report  we  shall  concentrate  on  but  one  aspect  of  that  question,  pilot  training,  and 
in  particular  training  for  first-time  transition  to  FMS  and  glass  aircraft. 
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II.  TRANSITION  TO  GLASS 


A.  INTRODUCTION 


In  this  chapter  we  shall  briefly  outline  the  problems  encountered  when  pilots  transition 
from  traditional  cockpits  to  glass  cockpits  for  the  first  time,  the  research  issues,  and  the 
practical  decisions  facing  the  airline  training  community.  We  shall  discuss  later  in  the 
report  possible  intervention  strategies  for  dealing  with  these  problems. 

Reviewing  the  literature  in  automation  and  training,  Wiener  noted  (1993a)  that  very 
little  has  been  written  on  the  broad  subject  of  training  pilots  to  fly  high  technology 
aircraft,  and  even  less  on  the  more  limited  topic  of  first-time  transition  to  glass.  The 
situation  is  still,  six  years  later,  about  the  same.  A  welcome  addition  is  Sherman’s 
dissertation  (1997),  in  which  he  brings  the  general  automation  literature  up  to  date. 

Also  of  great  value  are  Billings’  NASA  report  (1996)  and  book  (1997)  on  cockpit 
automation.  Billings  remarks  (1996,  p.  121-122):  “Training  must  be  considered  during 
the  design  of  all  cockpit  systems  and  should  reflect  that  design  in  practice.  Particular 
care  should  be  given  to  documenting  automated  systems  in  such  a  way  that  pilots  will 
be  able  to  understand  clearly  how  they  operate  and  how  they  can  best  be  exploited,  as 
well  as  how  to  operate  them.” 

By  the  late  1990s  the  aviation  community  became  more  concerned  about  training  for 
high  technology  cockpits,  largely  as  a  result  of  a  number  of  dramatic  accidents  and 
incidents  occurring,  first  in  Airbus,  later  in  B-757  aircraft  (Hughes  and  Dornheim,  1995). 
It  is  inevitable  that  following  an  accident,  especially  one  in  which  the  causes  include 
lack  of  understanding  of  autoflight  modes,  that  the  method  and  adequacy  of  pilot 
training  in  the  advanced  cockpits  will  be  questioned.  The  accident  occurring  in  1995  to 
American  flight  965,  a  757  on  approach  to  Cali,  Colombia  was  particularly 
incomprehensible,  perhaps  because  of  the  airline  involved.  American  enjoys  a 
reputation  of  leadership  and  uncompromising  quality  in  its  pilot  training. 

Over-Use  of  Automation? 


The  Cali  accident  exposed  to  the  public  some  of  the  hazards  of  autoflight,  and  much 
was  written  both  in  the  human  factors  literature  and  the  public  press  about  the 
presumed  over-use  of  automation.  We  will  discuss  over-  and  under-use  of  automation 
elsewhere  in  this  report.  But  the  accident,  and  the  reaction  in  the  press,  centered 
around  training.  Why  were  the  pilots  not  better  trained  to  use  the  proper  autoflight 
modes,  or  to  revert  to  manual  flight?  Why  did  they  not  make  use  of  the  information 
available  (e.g.  DME)?  We  are  mindful  of  Curry’s  (1985)  plea  for  “tum-it-off  training”, 
made  ten  years  prior  to  the  accident.  Criticism  of  training  that  over-emphasized 
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automation  was  coming  from  all  directions. 

Even  the  usually  conservative  Aviation  Week  and  Space  Technology  spoke  up 
editorially.  In  an  editorial  in  early  1996  ( AWST ,  February  12,  p.  66),  immediately 
following  the  Cali  accident,  under  the  title  “Failing  grade  for  FMS  training",  the  editors 
waded  into  the  controversy  in  the  first  sentence,  writing,  “The  training  of  airline  pilots  in 
the  use  of  flight  management  systems  (FMS)  is  clearly  inadequate,  and  airlines,  aircraft 
manufacturers  and  avionics  suppliers  should  get  together  to  pursue  better  solutions.” 
Not  the  usual  stuff  that  Aviation  Week  and  Space  Technology  editorials  are  made  of. 
Two  months  later  they  carried  an  article  by  Morrocco  (1996)  in  which  he  quotes  Terry 
P.  Newman,  a  senior  test  pilot  in  the  U.K.  Civil  Aviation  Authority  and  member  of  the 
FAA  automation  team  (Federal  Aviation  Administration,  1996)  as  complaining  that 
pilots  are  being  encouraged  to  make  use  of  automation  “at  every  possible  opportunity, 
particularly  the  autopilot,  because  it  can  do  a  better  job  than  you." 

The  effect  of  excessive  confidence  in  automation  has  been  noted  in  some  accidents 
where  the  crews  are  turning  to  the  autopilot  in  an  attempt  to  resolve  a  deteriorating 
situation.”  In  September  of  the  same  year  they  again  editorialized  under  the  heading 
“Training  is  no  band-aid  for  bad  design”  (AWST,  September  2,  1996,  p.  228),  stating, 
“Unfortunately  automation  has  neither  removed  human  error  nor  simplified  the  pilot’s 
job.  Instead  engineers  have  used  the  power  of  the  computer  revolution  to  cram  more 
functions  into  smaller  boxes,  more  information  onto  displays,  and  more  options  into 
flight  management  systems  than  the  average  pilot  has  any  hope  of  mastering.”  A 
comprehensive  report  by  Galante  (1995),  which  included  a  field  study  of  actual 
performance  on  the  line,  identified  reasons  why  pilots  “click  off’  the  autopilot,  or  certain 
flight  modes,  and  continue  with  a  lower  level  of  automation.  She  did  not,  however, 
relate  these  to  training.  The  following  NASA  ASRS  report  illustrates  the  concern  with 
over-reliance  on  automation. 


NARRATIVE:  Cruising  at  FL370  inbound  to  BDF  VOR  on  J 105  from  the  southwest. 
Kansas  City  Center  gave  us  clearance  to  cross  70  nm  south  of  BDF  at  FL330.  I  do  not 
clearly  recall  how  far  from  BDF  we  were  at  that  time.  But  we  immediately  began  to 
program  our  newly  fully  compliant  flight  management  computer  (FMC)  for  the  descent. 
We  twice  attempted  to  set  up  the  descent  using  the  full  FMC  capabilities  but  were  not 
successful,  so  we  then  reverted  to  the  more  basic  FMC  capabilities  and  were  in  the 
process  of  starting  descent  when  the  controller  inquired  if  we  were  going  to  be  able  to 
make  our  crossing  restriction.  He  added  that  we  had  only  9  miles  to  go.  I  immediately 
reverted  to  a  manually  controlled  descent,  i.e.  throttles  idle,  speed  brakes  deployed  and 
maximum  rate  of  descent.  We  told  the  controller  that  we  would  try  to  make  the  crossing 
restriction.  I  believe  that  several  factors  contributed  to  this  incident:  1)  this  was  my 
second  trip  after  being  off  this  aircraft  for  four  months  I  had  been  to  wide  body  recurrent 
training  in  Oct.  and  had  renewed  currency  in  the  simulator.  That  simulator  does  not  have 
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the  "full-up”  FMC.  2)  my  copilot  was  a  qualified  Captain  on  wide  body  who  was  on  his 
first  trip  after  requalifying,  4  months  since  his  last  wide  body  trip.  Both  pilots  are  dual 
qualified,  i.e.  simultaneous  qualification  on  other  equip.  3)  we  were  attempting  to  utilize 
the  new  full  up  features  of  the  FMC,  but  neither  of  us  were  proficient  in  its  use  Nor  had 
we  been  given  any  hands  on  training  on  the  new  features.  4)  we  had  earlier,  with  the  help 
of  a  written  text,  programmed  the  FMC  to  cross  BDF  at  FL240  and  were  not  expecting 
the  FL330  restriction.  5)  we  neglected  to  refuse  the  clearance  when  it  appeared  doubtful 
that  we  could  not  comply.  6)  we  both  allowed  ourselves  to  become  "mesmerized"  by  the 
computer  programming,  which  we  were  both  trying  to  learn  by  doing  7)  I  believe  I  suffer 
from,  as  I  believe  many  pilots  do,  a  reluctance  to  revert  to  basic  skills  and  methods, 
abandoning  the  advanced  technology  in  our  modem  aircraft  That  technology  seems  to 
lure  one  into  a  dependence  and  therefore  a  state  of  unwillingness  and  unpreparedness  to 
come  to  the  realization  that  operating  the  equipment  in  the  "real  world"  ATC  environment 
is  not  the  same  as  a  sterile  simulator.  This  impression  seems  to  me  to  be  reinforced  by  the 
"official"  insistence  that  the  technology  be  used  as  it  is  an  integral  and  essential  part  of  the 
two  man  crew  concept.  1  feel  that  we  have  neglected  to  emphasize  that  the  technology 
has  its  definite  limitations  in  this  real  world.  Had  I  been  more  prepared  to  override  the 
automatic  features  of  the  flight  guidance  system  I  feel  we  would  have  had  no  problem 
complying  with  the  clearance  ACCESSION  NUMBER:  59982 


So  at  least  part  of  the  training  agenda  has  been  defined  as  a  result  of  the  accidents 
and  incidents  of  the  first  half  of  this  decade,  and  the  field  studies  that  had  uncovered 
the  problems  even  earlier  (Curry,  1985;  Wiener,  1985b,  1988).  It  is  now  clear  that 
training  for  autoflight  must  include  not  only  proficiency  in  each  autoflight  mode,  but  also 
training  on  mode  selection  for  the  task  at  hand,  and  turn-it-off  training  as  well.  In  order 
to  achieve  this,  not  only  must  training  methods  and  curricula  be  modified,  but 
administrative  support  for  the  pilot’s  right  and  duty  to  use  or  not  use  the  automation  as 
he/she  sees  fit  must  be  clear. 

Aviation  automation  practitioners  and  researchers  should  note  that  we  are  not  alone  in 
recognizing  the  potential  problem  of  over-use  of  automation.  The  maritime  world  as 
well  suffers  from  presumed  over-reliance  on  automatic  devices.  A  brief  article  in 
Professional  Mariner  magazine  (December/January  1998,  pp.  68-69)  describes  the 
grounding  of  the  cruise  ship  Royal  Majesty  near  Nantucket  Island  in  June  1995.  The 
NTSB  determined  that  the  probable  cause  of  the  grounding  was  “over-reliance  on  the 
automated  features  of  the  integrated  bridge  system;  Majesty  Cruise  Line’s  failure  to 
ensure  that  its  officers  were  adequately  trained  in  the  automated  features  of  the 
integrated  bridge  system  and  in  the  implications  of  this  automation  for  bridge  resource 
management  [an  adaptation  of  CRM  for  ships];  the  deficiencies  in  the  design  and 
implementation  of  the  integrated  bridge  system  and  in  the  procedures  for  its  operation; 
and  the  second  officer’s  failure  to  take  corrective  action  after  several  cues  indicated  the 
vessel  was  off  course.”  All  of  this  language  should  sound  very  familiar  to  those  who 
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have  read  aviation  accident  and  incident  reports  involving  high  tech  aircraft 
Automation  Philosophy 

Relief  came  in  the  form  of  an  “automation  philosophy”  statement  (Wiener,  1985a), 
pioneered  at  Delta,  then  Continental  Airlines,  then  several  others.  The  Delta  statement 
appears  in  Wiener  et  al.,  1991 .  The  Continental  statement  and  its  development  are 
discussed  in  Chapter  VII  of  this  volume,  and  the  various  forms  of  the  automation 
statement  are  in  Appendix  F.  The  Delta  and  Continental  statements,  and  imitators  that 
followed,  say  essentially  the  same  thing:  the  pilot  must  be  proficient  in  all  autoflight 
modes,  but  the  selection  of  the  mode  or  modes  to  be  employed  (including,  presumably 
totally  manual  flight)  rests  with  the  crew.  There  are,  to  be  sure,  practical  limitations  on 
this.  Federal  Aviation  Regulations  (FARs)  require  the  use  of  certain  automatic  features 
for  low  visibility  approaches.  Pilot  discretion,  except  in  an  emergency,  stops  at  the 
doorstep  of  the  FAR. 

Chidester,  writing  of  American  Airlines’  approach  to  the  question  of  authority  to  select 
modes,  said 

“What  we  are  trying  to  establish  in  the  classroom,  and  through  this  article,  is  to  encourage 
our  pilots  to  develop  their  judgment  on  how  to  use  the  automation  on  their  aircraft.  Many 
pilots  report  feeling  pressured  to  always  operate  in  the  highest  mode  of  automation 
available.  We  need  to  remove  that  perceived  pressure  and  encourage  pilots  to  choose 
among  the  modes  in  any  given  situation.  To  do  that,  we  need  to  review  what  has  been 
automated,  some  of  the  documented  effects  of  automation,  and  some  lessons  learned 
(1994,  p.  8)  ” 

All  of  this  translates  into  a  training  requirement.  It  is  incumbent  on  the  training  syllabus 
to  ensure  the  first  requirement,  total  proficiency  in  all  modes,  and  to  instruct  as  well  on 
the  tactics  of  mode  selection.  The  first  task  is  relatively  easy  -  it  is  what  flight  training 
has  been  for  years,  only  now  with  modern,  extremely  flexible,  equipment.  The  second 
task  is  much  more  difficult.  Not  only  must  the  pilot  be  taught  discretionary  use  of 
autoflight  modes,  he  must  be  examined  and  graded  on  his  choices.  In  a  previous  field 
study  on  the  757  (Wiener,  1989),  during  an  interview  a  captain  had  this  to  say 
regarding  a  simulator  check  ride:  “All  my  life  the  FAA  examiner  has  been  turning  things 
off:  now  they  make  us  turn  everything  on.”  The  problem  of  autopilot  mode  errors  was 
first  pointed  out  by  Wiener  and  Curry  (1980),  and  has  been  a  popular  subject  for 
automation  researchers  (see  for  example,  Degani,  Shafto,  and  Kirlik,  1995;  in  press)  as 
well  as  the  operational  community.  As  we  have  previously  pointed  out,  autopilot 
modes  and  their  potential  for  human  error  has  been  discussed  by  many  authors,  so  we 
will  not  pursue  this,  except  to  say  that  this  is  a  major  area  of  concern  for  transition 
training,  both  in  the  pre-simulator  and  the  simulator  phase. 
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B.  RESEARCH  METHODS  IN  FLIGHT  TRAINING 


We  shall  note  briefly  here  the  various  research  methods  that  are  available  and 
appropriate  for  examination  of  pilot  training  for  and  transition  to  high  technology 
cockpits.  A  somewhat  more  detailed  presentation  can  be  found  in  Wiener  (1993a) 

Opinion  Surveys 

The  opinion  or  attitude  survey  is  widely  used,  due  to  its  relative  ease  of  administration 
and  analysis.  Wiener  (1993a)  listed  ten  studies  employing  attitude  measurement. 

Other  pilot  attitude  studies  have  since  been  published,  including  Sherman,  1997; 
Sherman,  Helmreich,  Smith,  Wiener,  and  Merritt,  1996;  Gras,  Moricot,  and  Poirot- 
Delpech,  1994;  Rogers,  Tenney,  and  Pew,  1995;  Tenney,  Rogers,  and  Pew,  1998; 
Sarter  (1991);  Sarterand  Woods  (1993,  1994,  1995);  Woods  and  Sarter  (1992); 
Madigan  and  Tsang,  1990;  and  Australian  Bureau  of  Air  Safety  Investigation  [BASI] 
(1998). 

Most  experimenters  have  been  content  to  display  the  results  of  each  attitude  probe  by 
tabular  or  graphic  means,  and  perhaps  test  certain  hypotheses  using  attitude  data  (as 
in  the  present  volume).  These  investigators  are  contrasted  with  those  who  have  done 
extensive  multivariate  analyses  on  their  data.  Two  examples  of  the  latter  approach  are 
the  work  of  McClumpha,  James,  Green,  and  Belyavin  (1991),  and  Sherman  (1997). 
Attitude  surveys  have  been  criticized  for  being  superficial,  and  not  obtaining  “real”  data, 
hard  performance  measures  that  one  would  prefer  in  human  factors  work.  A 
considerable  literature  has  developed  defending  attitudes  as  measurement  of 
performance;  as  reviewed  by  Sherman  (1997).  In  the  typical  survey  experiment,  the 
sample  sizes  tend  to  run  in  the  area  of  100  to  200.  Often  the  population  being  sampled 
is  small  by  definition  (e.g.  MD-11  pilots  going  through  transition  training  at  a  certain 
airline).  McClumpha  et  al.  (1991)  defined  a  larger  population,  European  pilots  from  a 
variety  of  aircraft,  leading  to  a  sample  of  572,  which  is  at  the  high  end  of  sample  sizes 
so  far.  Sherman  (1997)  made  use  of  the  vast  database  constructed  by  Helmreich  and 
his  colleagues  at  the  University  of  Texas,  with  sub-populations  in  the  thousands  (see 
Helmreich  and  Merritt,  1998;  and  Sherman,  Helmreich,  and  Merritt,  submitted  for 
publication). 

The  value  of  pilot  opinion  data,  when  hard  data  such  as  performance  measures  during 
a  simulator  run  might  be  preferred,  will  never  be  fully  resolved.  Opinion  survey  are 
above  all  easy  to  do  and  relatively  inexpensive.  In  contrast,  one  simulator  session  can 
cost  as  much  as  $2,400.  The  data  from  attitude  surveys  are  valuable  per  se,  for 
example  in  evaluating  a  hardware  design  or  a  training  method,  or  a  general  belief  such 
as  the  probe  used  in  this  study;  “I  have  no  trouble  staying  ‘ahead  of  the  plane’”.  As 
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many  authors  have  asserted,  attitude  data  can  be  taken  as  imperfect  measures  of 
system  performance. 


In-Flight  Observations 

Observations  taken  from  the  jumpseat  during  line  operations  are  the  ultimate  in  realism. 

Examples  are  Helmreich  and  Foushee,  (1993);  Helmreich  and  Merritt,  (1998);  Degani 

and  Kirlik  (1995);  and  an  extensive  study  of  mixed-line  flying  of  various  models  of  the 

B-737  by  Lyall  (1990). 

In-flight  observations  are  difficult  to  come  by  for  a  variety  of  reasons: 

1 .  In-flight  observation  requires  a  trained  observer,  familiar  with  the  aircraft 
systems,  air  traffic  control,  flight  regulations,  and  flight-deck  procedures.  Human 
factors  personnel  with  those  qualifications  are  rare.  Often  this  problem  is 
overcome  by  using  retired  pilots  as  observers,  which  creates  a  training 
requirement  of  its  own.  Former  pilots  may  be  familiar  with  the  environment,  but 
are  not  necessarily  good  observers,  and  their  expertise  in  human  factors  may  be 
modest. 

2.  Observing,  and  especially  taking  notes  or  logging  data,  may  be  frowned  upon  by 
the  crew.  It  is  one  thing  to  have  a  passive  observer  in  the  jumpseat;  it  is  quite 
another  to  have  someone  logging  data.  The  airline  cockpit  is  one  of  the  most 
exclusive  work  environments  known,  outside  of  government,  military,  or  law 
enforcement  operations,  and  this  is  jealously  guarded  by  those  who  work  there. 
Exclusive  or  not,  who  among  us  would  enjoy  having  someone  observe  our  work 
day,  occasionally  writing  something  on  a  clipboard  or  punching  keys  on  a  digital 
device  whenever  we  say  or  do  something,  or  perhaps  when  we  do  nothing? 

3.  Cockpit  observing  is  expensive,  and  possibly  inefficient.  It  is  efficient  in  that  the 
experimenter  does  not  have  to  build  or  buy  anything  -  the  “laboratory”  is 
furnished  by  the  airline,  ATC,  and  the  FAA.  The  inefficiency  comes  from  the 
paucity  of  occurrences  of  the  events  that  the  observer  may  be  looking  for,  e.g. 
TCAS  encounters,  certain  kinds  of  errors  in  using  the  automation,  or  perhaps 
CRM  behaviors  of  a  specified  type.  One  can  fly  many  legs  and  never  see  what  it 
is  that  he  is  looking  for,  since  it  is  usually  low  probability  events  that  are  of 
interest. 

4.  There  is  the  age-old  problem  of  observation  effects.  The  mere  presence  of  an 
observer  may  alter  the  behavior  of  the  crew.  This  is  difficult  to  overcome,  and 
can  probably  only  be  accomplished  by  long-time  exposure  with  the  same  crew. 
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Observation  of  Training  in  a  Simulator  or  Flight  Training  Device 


Both  the  simulator  and  the  lower  fidelity  flight  training  device  offers  a  highly  valid 
platform  from  which  to  observe  not  only  the  behavior  of  the  crew,  but  the  device  and 
the  instructor  as  well.  In  our  research  on  transition  to  glass,  these  observations  were 
invaluable.  Simulator  training  was  observed  in  many  field  studies,  including  Curry, 
1985;  Wiener,  1985b,  1989;  Wiener  etal.,  1991;  and  Sarter,  1991;  Woods  and  Sarter, 
1992. 


This  approach  has  some  of  the  same  problems  as  those  encounter  during  in-flight 
observations,  as  described  above.  However  there  is  clearly  little  or  no  observer  effect: 
the  instructor  absorbs  whatever  anxiety  there  is  about  being  observed.  The  human 
factors  observer  is  insulated  from  the  crew,  stationed  in  the  back  of  the  simulator  cab, 
out  of  sight,  out  of  mind. 

The  same  observations  that  we  made  with  respect  to  cost  and  difficulty  of  in-flight 
observations  apply  to  the  simulator  as  a  research  tool:  qualification  of  the  observer, 
cost  of  the  device,  and  rarity  of  event  if  the  observation  is  looking  for  something 
specific.  Usually  in  research  into  transition  to  a  higher  level  of  automation,  the 
observer’s  scope  is  wide-angle.  He/she  is  interested  in  almost  anything  that  reveals 
what  happens  when  pilots  move  from  low-  to  high-technology  cockpits. 

Experiments  in  Simulators 

The  simulator  offers  the  ideal  compromise  between  the  valid  but  uncontrolled  real  world 
of  line  flying,  and  the  highly  controlled,  but  far  from  valid,  experimental  booth.  The 
simulator’s  validity  is  extremely  close  to  the  Teal  thing,”  but  it  still  has  drawbacks  that 
the  experimenter  must  consider. 

1.  For  all  its  realism,  the  simulator  scenario  is  still  not  line  flying,  no  lives  and  no 
equipment  are  at  risk.  As  absorbed  as  simulator  pilots  may  be,  they  still  know 
that  they  are  in  a  box,  on  the  ground,  and  no  amount  of  simulated  ATC  chatter, 
weather,  electronic  visual  scenes,  or  motion  is  going  to  change  that. 

2.  For  the  human  factors  experiment,  the  extreme  realism  of  the  simulator  comes  at 
an  extreme  price.  In  one  simulator  study  of  automation  effects,  (Wiener,  et  al. , 
1991),  the  study  had  to  buy  simulator  time  (on  two  simulators  -  DC-9  and  MD- 
88),  instructor  time,  and  in  addition  pay  for  a  pilot-observer.  Fortunately  pilot 
volunteers  served  as  subjects  without  compensation. 

3.  Airline  simulators  are  not  equipped  for  human  factors  research.  Additional 
equipment  to  record  parameters,  sample  data,  and  record  pilot  inputs  may  be 
required.  (The  addition  of  closed  circuit  TV  cameras  in  the  simulator  cockpit,  to 
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facilitate  LOFT  debriefings,  has  been  a  boon  to  the  human  factors  researcher.) 
More  and  more  experimenters  are  turning  to  research  simulators  (such  as  those 
at  NASA  Ames  and  NASA  Langley)  for  their  work.  These  simulators  are  either 
built  ab  initio  to  provide  for  data  collection,  or  they  are  retrofitted  airline 
simulators. 

4.  Following  a  simulator  study,  the  experimenter  is  left  with  a  massive  data 

reduction  task,  long  before  statistical  analyses  can  be  performed.  It  may  take 
several  person-years  to  reduce  the  data  to  useable  form,  particularly  if  the 
variables  under  study  are  qualitative  (e.g.  quality  of  CRM  behavior)  rather  than 
flight  parameters. 

As  examples  of  simulator-based  experimentation,  we  recommend  Foushee,  Lauber, 
Baetge,  and  Acomb,  1986;  Wiener  et  al.,  1991;  T  Abbott,  1995;  Sarter  and  Woods, 
1995;  and  Veillette,  1995, 


C.  DIFFICULTIES  IN  TRANSITION  TRAINING  PROGRAMS 


The  emergence  of  the  glass  cockpit  brought  a  host  of  training  problems,  and  failure 
rates  in  transition  training  heretofore  unknown.  The  typical  failure  rate  that  was 
reported  in  the  early  and  mid-1980s  was  in  the  neighborhood  of  15  to  17  per  cent. 
Dornheim  (1992),  describing  the  development  of  MD-11  training  by  Douglas,  quotes  an 
a  most  unbelievable  figure  of  40  per  cent  failures  in  transition  to  the  MD-1 1  in  its  early 
days.  Typically  the  failure  rate  in  transition  training  to  various  models  with  conventional 
cockpits  has  been  less  that  one  per  cent.  Something  was  clearly  amiss.  After  a 
complete  redesign  of  training  programs,  and  the  investment  in  very  expensive  flight 
training  devices,  the  rate  was  brought  down  to  about  2  per  cent. 

Before  continuing,  we  should  take  note  that  the  alarming  rates  no  longer  exist,  and  the 
failure  rate  for  transition  to  glass  is  in  the  one  per  cent  range  (Wiener,  1993a, 

Dornheim,  1996b).  In  the  present  study,  only  two  pilots  from  the  original  sample  of  148 
failed  the  757  transition  course,  and  both  were  highly  unusual  cases  where  motivation 
and  personality,  not  the  pilots’  ability,  nor  the  quality  of  the  training  program,  was  the 
clear  explanation. 

We  shall  next  discuss  a  few  of  the  possible  causes  of  the  initially  disastrous  training 
situation  encountered  by  most  carriers  in  their  early  experience  with  transition  to  glass 
cockpit  technology. 
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Misinformation  and  Misconceptions 


Pilots  arriving  at  transition  training  often  came  with  a  stock  of  misconceptions.  There 
was  a  collection  of  bizarre  accounts  of  glass  aircraft  taking  over  from  helpless  pilots. 
One  such  story  that  swept  the  airline  community  tells  of  an  A-320  that  entered  a 
holding  pattern  and  could  not  get  out,  imprisoned  by  advanced  electronics!  It  did  not 
help  the  trainers  that  pilots  arrived  with  such  accounts,  mythical  as  they  were. 

Defeatist  Attitudes 


The  outrageous  stories,  along  with  rumors  (some  correct,  unfortunately)  of  high  failure 
rates,  led  to  pilots  arriving  for  training  with  attitudes  of  self  defeat.  They  had  also  heard 
that  the  program  was  impossible  --  the  popular  saying  that  ground  school  for  the 
transition  to  glass  was  like  the  proverbial  “drinking  from  a  firehose.”  At  other  airlines 
where  we  had  conducted  field  studies  in  the  decade  of  the  1980s,  it  was  said  (in  one 
form  or  another)  that  every  pilot  on  his/her  way  to  transition  training  had  a  neighbor 
who  had  a  cousin  who  worked  for  a  man  who  lived  next  door  to  a  pilot  who  had  washed 
out  of  glass  transition  training.  It  was  also  commonly  stated  that  the  older  captains 
could  not  pass  the  course,  due  to  their  lack  of  computer  familiarization,  and  perhaps 
due  as  well  to  the  general  prejudice  about  old  dogs  and  new  tricks.  It  is  little  wonder 
that  some  captains  showed  up  at  the  training  centers  with  an  over-powering  sense  of 
impending  defeat.  Many  withdrew  their  bid  and  returned  to  their  traditional  aircraft. 

Dornheim  (1992,  p.  93)  wrote  of  his  own  frustration  with  his  introduction  to  the  complex 
automation  of  the  MD-1 1 . 

The  simulator  session  gave  me  a  rude  awakening  about  the  realities  of  modem  glass- 
cockpit  aircraft.  I  expected  some  takeoffs,  landings,  approaches  to  stalls,  engine  failures 
and  other  maneuvers.  Instead,  I  received  a  frustrating  walkthrough  of  the  automatic  flight 
control  system  and  endless  complexities  of  the  flight  management  system  (FMS).  I  was 
irritated  at  first,  but  then  I  realized  that  this  was  what  it  was  all  about  —  pushing  buttons 
and  memorizing  FMS  screen  pages. 

Dornheim  continues  in  this  article  to  trace  the  design  and  even  the  costs  of  the  various 
training  devices  employed  by  Douglas  for  its  MD-1 1  training.  Costs  of  training  are  also 
discussed  in  Chapters  VIII  and  IX. 

Poor  Curriculum  Planning  and  Implementation 

Much  of  the  blame  lay  not  on  the  rumor  mill,  but  where  it  belonged,  on  those  who 
designed  the  training  syllabus.  Most  researchers  who  have  examined  this  area  agree 
that  the  basic  problem  was  that  the  early  curriculum  planners  were  hidebound, 
attempting  to  design  their  programs  as  if  they  were  training  pilots  for  the  727 
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The  first  author  attended  757  ground  school  at  two  major  airlines  in  1986.  At  one  in  the 
first  session  of  the  first  day  of  ground  school,  the  instructor  taught  the  class  CDU 
operations,  including  how  to  build  “man-made”  waypoints.  There  was  virtually  no 
introduction  to  the  airplane,  at  a  time  where  one  might  have  had  the  opportunity  to 
dispel  some  of  the  misconceptions,  and  ease  the  minds  of  the  students.  They  jumped 
immediately  into  the  most  difficult  and  unfamiliar  parts  of  transition  to  glass. 

Some  airlines  were  also  poorly  equipped  with  respect  to  training  hardware.  At  one 
airline,  ground  school  instruction  was  slide  projector  based,  but  they  did  not  have  the 
customary  cubical  and  projection  screen.  Projectors  and  pilots  were  lined  up  side-by- 
side,  four  to  a  table,  with  the  projectors  pointing  toward  the  wall.  At  any  given  time 
there  were  four  projected  images,  seldom  the  same,  on  the  wall  in  front  of  the  pilots, 
which  they  viewed  in  coordination  with  recorded  instruction. 

CRM  Taught  Separately 

The  1980’s  and  early  1990’s  initiated  the  era  of  CRM  training  (Wiener,  Kanki,  and 
Helmreich,  1993;  Foushee  and  Helmreich,  1988);  the  later  1990s  witnessed  the 
integration  of  CRM  with  conventional  flight  training  (systems,  maneuvers,  navigation, 
etc.)  This  concept,  pioneered  by  Boeing’s  flight  training  group,  was  the  result  of  earlier 
misdirected  effort,  leaving  the  student  with  the  notion  that  “real"  flight  training  and 
automation  training  were  one  area,  CRM  was  another. 

Boeing’s  contribution  was  to  show  that  the  two  were  inseparable  parts -of  flight  training, 
and  that  both  went  better  when  taught  and  practiced  as  an  integrated  whole.  This 
integration  of  the  two  formed  the  basis  for  CAL’s  training  program.  [More  is  said  of  this 
in  Chapter  VII.  See  also  the  quotation  at  the  top  of  page  VII-1 J.  From  the  first  day  in 
the  FTD  to  the  final  simulator  session,  procedures,  actions,  and  decisions  in  the  cockpit 
were  accompanied  by  communication  training  (CRM,  briefings,  etc.). 

Achievements 


One  by  one,  the  early  problems  of  transition  to  glass  have  been  solved.  The 
misinformation  has  abated.  Failure  rates  are  virtually  zero  in  the  transition  programs. 
Captains  and  older  first  officers  no  longer  have  distinguishable  difficulties  attributed  to 
their  age  or  computer  skills.  Training  program  curricula  have  been  vastly  improved  — 
no  more  warmed  over  727  lesson  plans.  CRM  has  been  integrated  into  flight  training, 
and  this  is  reflected  in  the  carriers’  AQP  applications.  Those  companies  that  offer 
introduction  to  automation  courses  early  in  ground  school  considered  them  a  great 
success. 

The  data  in  Figure  11-1  supports  the  impression  that  much  of  the  difficulty  had  been 
overcome.  The  data  come  from  the  first  questionnaire,  given  the  first  day  of  transition 
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training  for  the  757  pilots  (n=  148).  About  90  per  cent  of  the  pilots  either  reject  the 
probe  or  take  a  neutral  position.  About  ten  per  cent  accept  the  probe,  expressing  their 


8a  I  am  very  apprehensive  about 
going  through  this  transition 


<rongy  agr*« 


FIGURE  11-1.  Self-report  of  apprehension  upon  entering 
B-757  transition  program 


apprehension  about  the  transition  program.  We  feel  certain  in  earlier  days  of  glass 
cockpit  transition  programs  far  more  apprehension  would  have  been  reported.  And  to 
our  collective  relief,  there  are  as  yet  no  documented  cases  of  A-320s  (or  anyone  else) 
getting  stuck  in  a  holding  pattern,  beyond  the  control  of  the  flight  crew,  imprisoned  by 
their  automation,  and  destined  to  fly  all  turns  right,  two-minute  legs,  until  their  fuel  is 
exhausted  and  the  Law  of  Gravity  takes  over.  Such  a  story  today  would  bring  laughter 
where  it  once  brought  apprehension. 


D.  TRAINING  CONSIDERATIONS 


In  this  section  we  shall  discuss  some  of  the  factors  that  must  be  considered  in 
designing  and  implementing  a  transition  to  glass  program,  whose  ultimate  worth  will  be 
measured  on  the  line,  not  in  a  simulator.  We  will  consider  not  only  the  formal  design 
of  the  program,  but  various  human  factors  problems  encountered  in  automatic  flight, 
such  as  the  potential  for  skill  loss.  The  list  is  by  no  means  exhaustive.  The  factors  that 
now  must  be  taken  into  account,  particularly  when  operating  under  an  approved  AQP 
plan,  seem  endless.  This  is  one  of  the  virtues  of  the  AQP  process:  it  forces  the  training 
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department  of  an  airline  to  state  its  goals,  and  to  perform  a  detailed  analysis  of  the 
subject  matter,  as  well  as  the  teaching  and  learning  activities  required  to  reach  the 
specified  goals. 

Understanding  Autofliqht 

How  much  does  the  line  pilot  need  to  know  about  the  overall  flight  management  system 
of  the  757?  This  is  not  an  easy  question.  Should  the  pilot  merely  know  how  to  perform 
the  functions  he  wishes  to  use  in  flight,  or  should  he  have  a  larger  understanding  of  the 
overall  autoflight  system?  Sarter  (1991 )  is  critical  of  the  present  training  methods, 
criticizing  them  for  a  “bottom-up”  approach  which  tells  them  how  to  get  the  job  done,  but 
nothing  about  the  overall  plan  and  philosophy  of  the  flight  management  system  (“top- 
down”  approach).  She  argues  that  with  top-down  training,  the  pilot  would  be  better 
equipped  to  solve  unique  problems,  diagnose  automation  “glitches”,  and  avoid  illogical 
or  dangerous  mode  errors,  all  of  this  because  they  would  be  able  to  understand  the 
consequences  of  the  modes  selected,  and  other  actions  and  selections  in  using  the 
automation. 

There  is  no  simple  answer  to  the  problem  raised  by  Sarter.  The  pilot  must  be  trained  to 
obtain  the  desired  output  as  a  function  of  his/her  input  to  the  automation,  and  this  is 
bottom-up  training  by  any  standard.  Is  it  necessary  for  the  pilot  to  “understand”  the 
system?  Would  the  accidents  that  are  discussed  by  Hughes  and  Dornheim  (1995),  for 
example,  the  A-300-600  crash  in  Nagoya,  have  occurred  had  the  pilots  been  trained 
under  a  more  top-down  philosophy,  and  the  crews  better  understood  the  consequences 
of  their  choices? 

Hopkins  (1992)  states,  “Pilots  are  unanimous  in  their  opinion  that  training  for  the  ‘glass 
cockpit’  should  not  be  based  on  the  same  assumptions  which  form  the  framework  for 
conventional  flight-deck  training,  yet  it  still  is.”  He  goes  on  to  quote  Captain  Steve  Last, 
a  highly  experienced  pilot  and  trainer,  who  said,  “We  should  avoid  FMS  training  with 
insufficient  ‘overview’  at  the  start;  trainees  have  difficulty  later  in  synthesizing  the  detail 
to  see  the  whole.”  J.  Butler  (1991)  argues, 

“The  principles  of  training  for  advanced  technology  cockpits  are  not  dissimilar  to  those  of 
older  technology.  One  of  the  most  important  aspects  remains  to  select  the  right  people  for 
the  task  and  then  to  provide  the  necessary  hardware... and  training  devices  to  enable  a 
rapid  and  efficient  acquisition  of  knowledge  and  skill... The  fundamentals  of  the  aircraft 
operation  must  be  clearly  established,  understood  and  supported  by  all  instructors  and 
acquired  and  complied  with  by  trainee  pilots.  Training  courses,  whilst  necessarily 
concentrating  on  the  acquisition  of  flying  skills  must  also  place  great  emphasis  upon  the 
human  factors  aspects  of  team  work,  crew  co-ordination,  communication,  leadership, 
judgement  and  decision  making.” 
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It  is  tempting  to  say  that  line  pilots  must  not  only  be  able  to  operate  but  to  understand 
the  FMS.  But  our  corporate  memory  of  earlier  generations  of  aircraft  and  flight  training 
should  disturb  us.  It  was  not  too  long  ago  that  pilots  were  taught  “everything,”  including 
details  of  how  systems  worked,  specifications  and  limitations,  detailed  knowledge  about 
systems  over  which  the  pilot  had  no  control,  and  were  the  concern  only  of  maintenance 
workers.  With  the  coming  of  the  jet  age,  a  new  training  doctrine  arrived:  teach  the  pilot 
only  what  he/she  needs  to  operate  the  plane,  and  leave  the  rest  to  maintenance. 

Learning  details  of  electrical,  hydraulic,  and  pneumatic  systems  and  how  they  work 
together  is  not  the  same  as  understanding  the  inter-related  autoflight  modes  and  how 
they  work  together.  It  is  only  be  research,  and  observation  of  line  experience,  including 
accidents  and  incidents,  that  we  might  some  day  be  able  to  answer  the  top-down 
versus  bottom-up  question.  In  the  meantime,  the  training  departments  of  the  world 
must  strike  a  compromise  in  determining  just  what  level  of  detail  a  pilot  must  master. 

An  editorial  by  AWST  (February  12,  1996,  p.  66)  entitled,  somewhat  provocatively 
“ Failing  grade  for  FMS  training”  states,  “The  training  of  airline  pilots  in  the  use  of  flight 
management  systems  (FMS)  is  clearly  inadequate..."  The  editorial  goes  on  to  discuss 
the  fact  that  something  of  a  “cottage  industry”  has  grown  up  in  the  airline  pilot 
community.  This  industry  supplies  unofficial  manuals  covering  cockpit  automation. 

The  efforts  are  well  meaning,  but  not  welcomed  by  flight  management,  due  to  the  fact 
that  these  manuals  are  not  official,  are  not  approved  by  the  FAA  or  the  customer’s 
company.  And  they  may  contain  errors. 

Why  do  pilots  buy  these  products?  As  Orlady  (1991)  pointed  out,  pilots  are  never 
satisfied  -  they  will  always  say  that  they  need  more  training.  This  was  confirmed  in 
earlier  field  studies,  and  in  our  interviews  and  open-ended  questions  in  the  present 
study,  even  though  the  pilots  expressed  favorable  views  of  the  training  program. 

Skill  Degradation 

From  our  earliest  field  studies  to  the  present  we  have  heard  repeatedly  from  the  pilots 
in  training  for  glass,  or  in  their  first  years  of  flying  glass,  of  their  fear  of  skill  loss  due  to 
their  dependence  on  autoflight  (Curry,  1985;  Wiener  et  al,  1991;  and  Veillette,  1995). 
McClumpha,  James,  Green,  and  Belyvin  (1991),  in  a  large-scale  survey  of  attitudes 
toward  automation  of  European  pilots,  also  found  the  same  concern,  and  reported  as 
well  that  older  pilots  (>  50  years)  and  those  with  more  training  hours  showed  less 
concern.  Skill  loss  is  a  realistic  issue.  No  pilot  wants  to  stand  by  and  watch  the  asset 
he  values  the  most,  his  flying  skill,  deteriorate.  At  some  companies,  where  policies 
required  use  of  autoflight  modes  whenever  possible,  the  concern  multiplied.  Pilots 
found  some  relief  at  those  companies  which  developed  automation  guidelines  such  as 
those  derived  at  Delta  and  Continental,  allowing  the  pilots,  under  most  circumstances, 
a  choice  of  autopilot  modes  (including  no  autopilot  -  hand  flying). 
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Many  pilots  adopted  their  own  code  of  hand  flying.  For  example,  when  they  were  pilot 
flying  (PF),  after  takeoff  they  would  hand  fly  to  the  first  level-off  altitude,  then  engage 
the  autopilot;  or  perhaps  hand  fly  to  cruise  altitude.  Some  would  hand  fly,  with  or 
without  the  flight  director,  at  least  one  ILS  approach  per  trip,  weather  permitting.  The 
list  is  endless.  The  self-imposed  rules  were  taken  seriously,  almost  as  if  they  were 
regulations.  In  a  previous  field  study  of  B-757  pilots,  Wiener  (1989)  reported  that  90 
per  cent  of  his  questionnaire  respondents  stated  that  they  tried  to  hand  fly  some  portion 
of  each  trip,  and  if  possible,  some  portion  of  each  leg 

Veillette  (1995)  states  four  reasons  that  skill  maintenance  is  important: 

1 .  Manual  flying  skills  are  necessary  to  handle  the  critical  flight  regime  of  the  jet. 

2.  Crews  who  become  task  saturated  in  terminal  areas  often  revert  to  manual 
flying.  [This  is  a  well  established  behavior  that  we  have  seen  in  other  studies, 
both  in  interviews  and  questionnaires,  as  well  as  in  jumpseat  observations  on 
the  line.  It  should  be  a  discomfort  to  those  who  claim  that  automation  implies 
workload  reduction.] 

3.  Some  ATC  clearances  require  a  high  degree  of  manual  skill,  if  they  are  not  done 
using  autopilot  modes  (e.g.  “slam-dunk”  approaches). 

4.  Manual  handling  of  the  aircraft  provides  information  and  situational  awareness 
to  the  pilot.  It  enhances  feedback  from  machine  to  pilot. 

ETOPS  operations  have  also  had  an  impact  on  automation  usage,  and  the  skills 
maintenance  issue.  Most  companies  which  operated  twin  engine  (usually  wide  body) 
jets  such  as  the  B-767  and  the  A-310  on  trans-oceanic  flights  also  used  the  equipment 
for  domestic  legs.  Some  segregated  the  pilots  who  were  flying  the  same  aircraft  into 
two  sub-fleets,  ETOPS  and  domestic.  Those  on  the  domestic  flight  had  far  less 
concern  about  skill  loss.  The  ETOPS  pilots  expressed  fear  that  they  were  losing  not 
only  manual  skills,  but  automation  skills  as  well,  since  almost  all  of  their  time  was  spent 
at  cruise,  with  few  opportunities  to  exercise  automation  skills,  particularly  CDU 
programming.  Some  ETOPS  trips  had  a  domestic  leg  at  the  end.  Pilots  welcomed  this 
as  an  opportunity  to  practice  automation  skills.  At  the  companies  that  allowed  pilots  to 
bid  both  types  of  trips,  pilots  who  were  concerned  about  skill  loss  would  typically  bid  a 
line  that  contained  two  overseas  trips,  and  several  days  of  domestic  flying  in  a  month. 
Again,  they  took  this  self-imposed  discipline  seriously.  This  may  no  longer  be  a 
problem.  Most  carriers  that  we  are  aware  of  allow  mixing  domestic  and  trans-oceanic 
trips.  Further  discussion  of  the  role  of  company  policies,  procedures,  and 
implementation  can  be  found  in  Orlady  (1991),  Chidester  (1994),  and  Degani  and 
Wiener  (1994). 
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Unfortunately  there  has  been  almost  no  experimental  work  on  the  topic  of  skill  loss  in 
today’s  automated  aircraft,  the  one  exception  we  are  aware  of  being  the  study  by 
Veillette  (1995).  What  we  know  about  the  subject  comes  from  interviews  and 
questionnaires.  Thus  it  is  difficult  to  design  an  intervention  if  we  do  not  know  the 
magnitude  or  locus  of  the  (presumed)  problem.  It  would  be  difficult,  even  if  a  simulator 
were  available,  to  plan  and  execute  such  an  experiment. 

“Backward  Transition” 


Backward  transition  refers  to  the  transfer  of  a  pilot  from  a  glass  environment  to  a 
traditional  cockpit,  and  whatever  problems  this  may  present.  Usually  the  backward 
transition  requires  only  one  or  two  days  of  formal  training  and  a  simulator  check  ride. 
The  presumption  is  that  a  pilot  who  has  been  flying  glass  for  some  time  may  encounter 
difficulties  if  he/she  returns  to  traditional  instruments.  Once  again  we  have  little  in  the 
way  of  data:  only  interviews,  attitude  surveys,  and  open-ended  questions.  (See 
Wiener,  1989,  pp.  87-92.)  We  are  not  aware  of  any  research  that  directly  deals  with 
backward  transition.  The  general  sense  of  what  the  pilots  said  was  that  they  had 
trouble  at  first,  but  very  quickly  overcame  it,  and  within  a  trip  or  two  were  up  to  the  level 
of  proficiency  that  they  enjoyed  prior  to  their  757  transition. 

The  biggest  difficulty  in  the  backward  transition  from  the  757  appeared  to  be  the  loss  of 
the  HSI  map  mode  display.  Pilots  in  Wiener’s  field  study  (1989)  had  expressed  a  great 
attachment  to  that  instrument,  and  for  better  or  worse,  they  had  learned  to  depend 
heavily  on  it  in  the  757.  The  problem  emerged  in  the  need  to  integrate  the  information 
from  various  displays  to  determine  one’s  position  What  pilot  would  not  miss  this 
display?  While  he  was  training  to  fly  glass,  it  made  navigation,  planning,  and  weather 
avoidance  so  simple  and  so  precise. 

Specifically,  the  pilots  felt  at  a  loss  without  the  map,  and  found  it  difficult  to  stay  “ahead 
of  the  airplane”  without  this  display.  Many  said  that  within  a  terminal  area,  either  on 
takeoff  or  approach,  if  ATC  turned  them  off  of  the  published  STAR  or  SID,  they  had 
trouble  taking  into  account  the  various  navigation  displays  in  the  traditional  aircraft  and 
knowing  where  they  were.  It  had  been  so  easy  in  the  757!  Also  mentioned  was  the 
ease  with  which  radar  and  navigational  information  could  be  combined  and  displayed 
on  the  757  map.  On  the  traditional  displays,  they  had  to  extract  radar  information  and 
then  mentally  combine  it  with  HSI,  DME,  ADF,  and  VOR  displays.  Again,  all  of  this  was 
“done  for  them"  by  the  glass  displays. 

In  fact,  newly  transitioned  pilots  had  to  be  told  not  to  stare  at  the  map,  but  to  bring  it 
into  their  scan  like  any  other  instrument.  There  should  be  little  surprise  that  line  pilots 
would  find  this  to  be  the  feature  that  they  missed  the  most  when  they  took  backwards 
transition  to  the  727s,  the  DC-9s,  and  the  747s  (Wiener,  1989). 
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Backward  transition  is  a  topic  that  is  interesting  to  discuss,  but  probably  not  very 
important.  Pilots  generally  do  not  see  it  as  a  problem,  at  least  not  after  a  day  or  so 
back  in  the  traditional  cockpit.  Whatever  problems  are  brought  about  by  backwards 
transition  seem  to  vanish  quickly,  and  problem  or  not,  the  whole  issue  will  disappear  in 
the  years  ahead,  when  there  is  no  primitive  cockpit  to  go  back  to  Were  it  not  for  the 
supernatural  longevity  of  the  B-727,  many  airlines  would  be  all  glass  by  now,  or  close 
to  it. 

We  raise  one  more  issue  before  leaving  backward  transition  --  the  possible  loss  of 
automation  skills  during  the  period  the  pilot  was  re-assigned  to  traditional  cockpits.  At 
some  point  this  pilot  would  return  to  glass  and  would  be  expected  to  have  lost  some  of 
his  knowledge  and  skills  in  operating  in  an  autoflight  environment.  The  question  for 
training  departments  is  how  much  re-training,  and  what  kind,  is  required  to  bring  the 
pilot  back,  hopefully  close  to  the  level  he  had  reached  in  his  original  glass  training. 

Ab  Initio  Training  and  Very  Low-time  New  Hires 

In  the  latter  part  of  the  decade  of  the  1980’s  airline  service  and  aircraft  began  to 
expand,  and  military  flight  training  began  to  diminish.  The  military  service,  the 
traditional  source  of  airline  pilots,  looked  to  be  insufficient  for  the  years  ahead, 
particularly  in  Europe  and  Asia.  Other  sources,  such  as  flying  academies,  could  not  fill 
the  gap.  So  major  airlines  in  both  European  and  Asian  countries  proposed  ab  initio 
(from  the  beginning)  training,  whereby  young  men  and  women,  entering  the  program 
with  zero  flight  time,  would  be  trained,  usually  at  airline  expense,  up  to  a  point  where 
they  had  their  basic  licenses  (see  Glines,  1990,  and  Telfer,  1993;  this  topic  is  also 
discussed  in  Chapter  VIII).  At  this  point  students  would  have  approximately  200-300 
flight  hours,  and  large  amount  of  jet  simulator  time.  They  would  then  go  through  type 
training  and  join  the  line.  The  type  could  be  whatever  the  airline  flew,  including  heavy 
jets  and  glass  cockpits. 

To  the  traditional  pilot  or  instructor  it  may  be  difficult  accept  that  a  low-time  pilot  trainee 
just  out  of  “primary  training"  could  occupy  the  right  seat  in  an  airliner.  In  the  U.S.  there 
was  concern  about  pilot  shortages,  but  ab  initio  training  was  never  a  very  attractive 
solution.  One  fleet  manager  told  us  that  he  doubted  that  CAL  would  ever  have  to  hire 
ab  initio  pilots.  CAL  hired  880  pilots  in  1998,  and  their  mean  total  flying  time  was  over 
3000  hours.  Each  had  turbine  time.  CAL,  like  many  carriers  with  their  own  commuter 
airlines,  will  draw  most  of  their  new  hires  from  their  commuter.  In  the  short  run,  CAL 
will  obtain  100  per  cent  of  their  pilots  from  their  commuter  ranks.  Later  there  will  be  a 
mixture  of  backgrounds. 

The  anticipated  pilot  shortage  never  occurred,  primarily  because  there  are  various 
ways  in  which  a  young  pilot  can  qualify  for  an  airline  seat  in  the  U.S.  A  300-hour  pilot 
would  have  a  difficult  time  finding  employment  at  even  the  smallest  airlines. 
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Inexperienced  pilots  in  the  U.S.  have  a  hard  life:  they  must  take  any  kind  of  flying  job 
(usually  instructing  beginning  students)  in  order  to  build  up  their  hours.  The  next  step 
is  usually  the  air  charter  or  Part  135  operator,  which  serves  as  a  “farm  club"  for  the 
larger  regional  carriers  and  the  major  airlines.  How  long  he/she  stays  at  a  farm  club 
depends  on  the  market.  There  is  some  movement  toward  the  pilot  applicant,  not  the 
airline,  to  carry  the  cost  of  primary  training.  Some  carriers,  such  as  Southwest,  require 
new  hires  to  have  a  type  rating,  in  this  case  for  a  737.  The  airline  hiring  the  pilot  may 
provide,  at  a  price,  the  necessary  training,  or  the  candidate  may  go  elsewhere  to  obtain 
his/her  type  rating 

Finally,  let  us  consider  the  following  question.  Given  a  low-time,  zero  jet  time  ab  initio 
graduate  who  is  recruited  by  an  airline.  If  the  airline  has  its  choice,  where  should 
he/she  be  placed  for  the  first  line  experience,  the  traditional  cockpit  or  the  automated 
model?  The  traditional  cockpit  is  simpler,  and  more  like  the  aircraft  the  applicant  had 
trained  in.  Remember,  his/her  exposure  to  autoflight  is  almost  nil.  The  737  or  DC-9 
sounds  just  right.  On  the  other  hand,  it  could  be  argued  that  the  very  low-time  pilot  is 
best  off  in  a  highly  automated  plane,  with  a  sophisticated  autopilot  and  autothrottle 
supporting  him.  Only  line  experience  and  research  will  answer  the  question  that  we 
have  posed. 

Airline  training  departments  may  find  it  difficult  to  believe  that  they  will  ever  hire  ab  initio 
or  other  very  low-time  pilots.  But  who  would  ever  have  dreamed  that  over  half  of  the 
graduating  class  from  the  Air  Force  Academy,  due  to  the  cutback  in  flying,  will  now  go 
to  non-flying  jobs? 

Advanced  Maneuvers  Training 

Following  a  series  of  airline  mishaps  and  close  calls  in  the  latter  half  of  the  1990's,  in 
which  the  aircraft  became  severely  upset  and  had  to  be  recovered  from  an  abnormal 
position,  some  airlines  instituted  “advanced  maneuvers”  simulator  training  for  all  pilots. 
Military  pilots  for  the  most  part  had  such  training  (traditionally  called  “unusual 
positions”),  but  those  who  came  to  the  airlines  from  civilian  sources  often  did  not.  This 
training  is  required  at  Continental.  For  a  comment  from  a  pilot,  see  the  ASRS  report  at 
the  end  of  this  chapter.  [Note:  advanced  maneuvers  are  not  part  of  this  study,  as  the 
program  came  after  our  work  at  Continental  was  complete]. 

Training  Devices 

We  shall  mention  training  devices  only  briefly,  as  so  much  has  been  written  on  this 
subject.  The  last  two  decades  has  witnessed  a  rapid  development  of  training  devices, 
both  at  the  high  end,  the  full  flight  simulators  (FFS)  and  what  will  probably  soon  be  the 
middle  on  the  sophistication  scale,  the  flight  training  device.  The  FTD  provides  pilot 
trainers  with  a  device  with  full  systems  and  flight  simulation,  including  autopilot  and 
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flight  director  modes  and  glass  displays.  In  the  interest  of  economy  of  both  purchase 
price  and  maintenance,  it  does  not  have  a  visual  scene  or  a  motion  base.  FTDs  offer 
the  pilot  trainee  an  excellent  platform  upon  which  to  obtain  cockpit  familiarization, 
including  running  checklists,  cockpit  procedures  (normal  and  abnormal),  flight 
maneuvers,  and  autoflight  modes.  For  a  discussion  of  the  importance  of  the  FTD  in 
one  program  (MD-11),  see  Dornheim  (1992). 

The  biggest  problem  is  that  FTDs  are  coming  at  a  steadily  increasing  price,  due  largely 
to  the  number  and  complexity  of  autoflight  modes  that  they  must  simulate.  One  airline 
that  participated  in  an  earlier  field  study  (Wiener,  1989)  found  that  their  FTD,  a  very 
elaborate  model,  was  converging  in  price  on  the  FFS.  They  canceled  their  order  for  a 
second  FTD,  preferring  to  put  their  funds  toward  a  second  FFS.  Better  to  pay  more  up 
front  in  order  to  have  the  sophistication  and  regulatory  status  that  only  a  full  flight 
simulator  enjoys,  they  reasoned. 

At  Continental  the  757  FTD  (Level  5  out  of  7  on  the  FAA’s  rating  scale  at  this  writing) 
and  the  computer  based  training  (CBT)  were  carefully  integrated  into  a  logical  syllabus. 
The  typical  ground  school  day  is:  two  hours  instructor  briefing,  two  hours  FTD,  and 
four  hours  CBT.  After  two  weeks  of  this,  they  move  to  the  FFS.  (At  this  writing, 
Continental’s  FFS  is  level  C  on  the  FAA  scale,  soon  to  be  upgraded  to  level  D,  the 
highest  level).  Training  emphasizes  not  only  the  technical  material  that  had  to  be 
mastered,  but  also  checklists,  procedures,  communications,  briefings,  and  CRM. 

What  seems  to  be  missing  in  the  array  of  training  devices  would  be  a  device  so  small 
and  so  inexpensive  that  it  could  be  provided,  along  with  the  software,  to  each  pilot,  not 
only  for  transition  training,  but  for  recurrent,  and  for  incorporation  of  new  devices.  One 
can  recall  the  confusion  that  existed  over  the  training  for  TCAS,  and  the  argument 
about  whether  TCAS  training  had  to  be  in  a  simulator.  We  asked  in  a  previous  field 
study  (Wiener,  1989)  why  the  personal  computer,  perhaps  as  a  home  study  aid  and 
motivator,  could  not  be  used  to  relieve  the  load  on  the  training  center,  and  particularly 
on  the  FTD.  This  subject  is  also  discussed  in  Chapter  VIII. 

Recently  there  has  been  some  developmental  work  on  using  an  ordinary  laptop 
computer  as  a  flight  training  device.  Stephen  Casner  of  NASA-Ames  has  programmed 
a  relatively  inexpensive  laptop  as  a  B-737  CBT  device,  with  highly  attractive  color 
graphic  displays.  Nordwall  (1995)  describes  how  the  U.S.  Navy  is  using  laptops  for 
pilot  training.  He  writes  (pp.  68-69),  “The  capability  of  the  new  CDNU  (control  display 
navigation  unit)  exceeded  Navy  expectations.  Its  use  has  broadened  from  a  tutorial  aid 
to  something  pilots  can  use  for  proficiency  training  and  dynamic  simulation.” 

Clearly  the  potential  exists  for  development  of  very  sophisticated,  low-cost  personal 
computer-based  devices.  The  problem,  as  always,  will  be  the  cost  of  development  and 
distribution  of  software.  One  could  envision  software  upgrades  being  included  in  the 
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pilots’  Jeppesen  revision  envelopes  Certification  of  PC-based  training  software  is 
something  that  the  FAA  presently  is  not  well  equipped  to  do.  Presently  there  is  also 
the  beginning  of  a  discussion  of  "web-based  training",  whereby  software  can  be  down¬ 
loaded  from  the  world-wide  web  (WWW)  to  a  personal  computer.  This  has  two 
advantages  over  conventional  PC-based  systems:  the  cost  of  delivering  software  is 
reduced;  and  the  uniformity  of  software  is  assured.  Web-based  learning  combines  the 
advances  of  PC-based  training  with  a  highly  efficient,  quality  controlled  means  of 
delivering  and  updating  software. 

Cost  of  Training 

We  shall  mention  only  briefly  the  matter  of  cost  of  training  See  also  Chapter  VIII.  We 
have  previously  discussed  the  rising  cost  of  flight  training  devices  and  simulators,  but 
simulators,  while  dramatic,  are  only  a  part  of  the  picture.  Cost  considerations  in 
designing  a  training  program  cannot  be  ignored,  in  the  highly  competitive  and  cost 
conscious  economic  environment  of  post-deregulation  operations.  Gone  are  the  days 
when  the  word  training  was  sacred,  and  training  departments  could  get  anything  they 
wanted  by  waving  the  flag  of  safety.  Today  every  cost  in  the  training  process  must  be 
justified,  and  the  justification  may  be  a  traditional  one  drawn  from  the  corporate  world, 
return  on  investment  (ROI).  Kelly,  Graeber,  and  Fadden  (1993)  discuss  the  ROI 
principle  in  flight-deck  design:  “While  many  operational  features  provided  by  a  flight 
deck  may  be  considered  desirable,  the  market  increasingly  demands  return  on 
investment  for  capabilities  as  opposed  to  features  (p.  56)”  The  same  statement  could 
be  made  about  training  capabilities,  though  the  direct  connection  to  the  market  place  is 
somewhat  less  visible.  Cost  figures  are  seldom  published.  An  exception  is 
Dornheim’s  article  (1992)  on  the  MD-11  school  at  Douglas. 

Orlady  (1991,  p.  2.6)  cautions  us  about  assuming  that  automation  can  reduce  training 
requirements:  “Unfortunately,  one  of  the  great  myths  of  automation  is  that  automation 
reduces  training  needs.  One  of  the  persuasive  arguments  for  further  use  of  automation 
has  been  that  it  reduces  training  costs.  This  assertion  is  patently  false,  particularly  in 
the  areas  of  manual  skills,  system  knowledge,  and  the  logic  of  the  automatics.” 

On  the  other  hand,  Leonard  (1993,  p.  149)  states  that  when  CBT  training  for  the 
advanced  cockpit  is  combined  with  an  FTD,  “The  results  have  been  an  overall  cost 
reduction  in  flight  crew  transition  training  and  an  increase  in  successful  training  rates 
for  advanced  flight  deck  aircraft.”  He  quotes  failure  rates  of  training  for  the  glass 
cockpit,  in  1984,  as  40  per  cent.  He  describes  the  problem  in  economic  terms,  saying 
(p  150),  “This  failure  rate  was  unacceptable  because  extensive  remedial  training  of 
flight  crews  was  economically  unacceptable.  The  high  failure  rate  dramatically 
highlights  the  inadequacy  of  existing  training  strategies  to  develop  the  cognitive  skills 
required  by  evolving  aircraft  technologies."  Leonard  goes  on  to  describe  the 
development  of  the  MD-1 1  training  package. 
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Crew  Resource  Management  (CRM) 


We  will  only  mention  CRM  training  briefly  here,  as  Chapter  VII  is  dedicated  to  CRM  at 
Continental.  Crew  resource  management,  or  alternatively  cockpit  resource 
management,  was  first  explored  by  the  airlines  in  the  1970's,  and  was  developed  as  a 
commercial  product  and  sold  to  other  airlines  by  United.  Not  until  the  early  1980's  was 
it  widely  used  or  explored  (Cooper,  White,  and  Lauber,  1979;  Wiener,  Kanki,  and 
Helmreich,  1993).  CRM  training  was  not  an  FAR  requirement,  but  its  presence  as  an 
FAA  advisory  circular  (AC  120-51 B)  is  a  clear  signal  to  the  air  carriers  that  the  FAA  has 
more  than  a  casual  interest  in  this  form  of  training.  The  AC  is  usually  a  precursor  of  an 
FAR.  Under  the  Advanced  Qualification  Program  (AQP),  CRM  is  part  and  parcel  of  the 
training  and  evaluation.  Not  only  must  CRM  be  included  in  the  training  proposal,  but 
pilots  will  be  evaluated  on  their  CRM  skills.  Continental  was  one  of  the  pioneers  of 
CRM  training;  their  training  approach  and  materials  have  been  widely  imitated. 

As  we  have  mentioned  elsewhere  in  this  report,  the  trend  today  is  toward  integrating 
CRM  training  and  technical  flying  training.  This  approach  has  been  developed  and 
encouraged  by  the  mainframe  manufacturers,  and  was  employed  by  the  757  planners 
at  Continental  in  their  design  of  the  training  syllabus.  Under  the  new  approach,  no 
longer  will  pilots  be  exposed  to  stand-alone  CRM  training  in  the  classroom.  CRM  will 
be  taught  in  the  FTDs,  the  simulator,  and  in  conversation  with  the  instructors,  as  a 
subject  intermingled  with  traditional  maneuver  and  procedure  training.  In  each 
maneuver,  or  checklist,  the  CRM  aspects  will  be  taught  along  with  the  technical 
training.  AQP  programs  also  encourage  this  type  of  training.  More  will.be  said  of  this 
in  Chapter  VII.  The  importance  of  learning  and  practicing  good  CRM  skills  can  be  seen 
in  the  ASRS  report  below. 


NARRATIVE:  The  problem  began  approx  100  nm  south  of  DC  A.  The  captain  was 
flying.  I  obtained  the  ATIS,  LDA-DME  18  was  in  use.  That  approach  is  not  in  our 
FMS's  database  so  I  started  to  build  it.  The  captain  told  me  not  to  do  that  His 
explanation  was  that  is  one  of  our  simulator  scenarios,  to  check  on  CRM  ability,  and  he 
wanted  to  practice  it  first.  About  5  mins  later  I  stated  my  concerns  about  not  using  all  the 
equipment  at  our  disposal.  The  navigation  display  is  a  great  help  and  we  were  not  going 
to  use  it.  The  captain  restated  his  wishes  and  I  dropped  it.  While  being  vectored  for  the 
approach,  I  idented  the  localizer,  however  we  were  not  receiving  the  DME.  As  we  were 
being  turned  onto  final  the  captain  instructed  me  to  reconfigure  my  panel  to  get  the  DME. 
This  leaves  me  with  no  localizer  indication  and  no  navigation  display.  By  the  time  I  did 
this,  we  received  another  turn  and  a  descent,  3000  ft  down  to  2000  ft  Then  I  noticed  the 
autopilot  was  not  set  to  capture  the  localizer.  I  pointed  this  out.  The  captain  armed  the 
autopilot.  He  is  new  (3  months)  to  the  airplane  and  was  behind.  As  we  descended 
Approach  asked  if  we  had  final  OK.  The  captain  lied  and  said  'yes. 'just  then  we  broke 
out.  I  saw  the  river  to  the  right  and  pointed  it  out.  Approach  once  again  asked  if  we  had 
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the  final  approach  course  and  gave  us  a  turn  to  the  right.  That  was  very  quickly  changed 
to  a  left  turn  and  climb  to  3000  ft.  We  accomplished  this,  did  the  after  takeoff  checklist 
and  followed  more  vectors  My  situational  awareness  was  shot.  I  offered  once  again  to 
build  the  approach  but  the  captain  refused.  We  were  once  again  vectored  to  final  and  I 
asked  Approach  what  was  our  relationship  to  the  localizer.  We  were  already  through  it 
and  getting  worse  Approach  Control  broke  us  off  once  again  for  the  VOR  18  We  asked 
for  time  to  review  it  and  set  it  up.  This  approach  was  successful.  There  was  no  CRM. 
The  equipment  on  board  was  not  properly  utilized  and  I  was  not  properly  utilized.  No 
matter  how  much  CRM  training  is  given,  some  people  don't  get  it  at  the  most  basic  level 
ACCESSION  NUMBER  110413. 

E.  CONCLUSIONS 


What  can  we  conclude  at  this  point  in  the  report?  It  is  clear  that  many  of  the  problems 
that  were  experienced  in  the  early  years  of  the  glass  cockpit  have  been  overcome.  As 
we  noted,  the  failure  rate  today  in  first-time  glass  transition  is  about  the  same  as  in 
traditional  aircraft,  less  that  one  per  cent.  But  pass/fail  does  not  tell  us  the  entire 
picture.  We  must  certain  that  those  99+  per  cent  who  graduate  and  transition  to  glass 
do  so  with  the  training  that  will  serve  them  well  in  their  line  flying.  The  line  is  the 
ultimate  test. 

The  most  striking  criticism  charges  that  flight  training  is  not  governed  by  any  overall 
philosophy.  Perhaps  the  final  product  is  an  amalgam  of  philosophies,  some 
complementary,  some  antagonistic.  It  is  essential  today,  and  will  be  more  essential  in 
the  future,  that  the  training  package  for  any  aircraft  be  consistent  not  only  with  the  best 
operation  of  the  equipment,  but  with  the  objectives  of  the  company  and  the  training 
objectives  of  the  entire  fleet.  With  a  variety  of  aircraft  flown  as  a  common  fleet  (e.g.  the 
many  models  of  the  B-737  that  are  presently  available  and  will  soon  be  on  the  line),  a 
unified  training  philosophy  is  essential. 

As  Degani  and  Wiener  (1994)  observed,  procedures  are  not  strictly  determined  by  the 
hardware:  the  same  piece  of  equipment  is  operated  according  to  different  procedures 
at  different  carriers.  The  procedures  are  governed  only  partly  by  the  hardware,  but 
also  by  the  philosophy,  background,  mission,  history,  operations,  and  corporate  culture 
of  the  company.  Some  differences  are  trivial  (e.g.  various  ways  to  set  up  TCAS 
modes),  some  are  dramatic.  For  example,  at  the  beginning  of  the  study,  only  one  major 
U.S.  airline  that  we  are  aware  of  employed  QFE  altimeter  procedures.  What  was  it 
about  the  flight  culture  of  that  company  that  they,  and  they  alone,  found  it  desirable  to 
use  QFE  altimetry?  In  1998  the  company  abandoned  the  use  of  the  QFE  altimeter 
(personal  communication,  T.  Chidester,  1999). 
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Degani  and  Wiener  interviewed  top  flight  management,  starting  with  the  vice-president 
for  flight,  at  three  major  airlines  on  procedure  development,  and  asked,  “why  do  you 
develop  your  own  procedures?  Why  not  just  follow  the  Boeing  (or  any  other 
mainframer’s)  procedures?”  The  answer  was  always  the  same:  "Boeing  designs  and 
assembles  aircraft.  We  fly  passengers.” 

We  believe  the  4P’s  model  of  Degani  and  Wiener  could  be  used  profitably  in  designing 
training:  AQP  may  have  already  forced  the  issue.  Their  model  states  that  philosophy 
determines  policies,  policies  lead  to  procedures  (or  in  the  matter  at  hand,  training 
packages),  and  procedures  are  compared  to  practices  (what  actually  occurs).  Without 
the  unifying  influence  of  the  first  two  P’s,  training  programs  are  likely  to  be  a  hodge¬ 
podge.  A  philosophy-based  training  program  could  avoid  this,  and  meet  the  critics’ 
charges  that  most  training  programs  are  based  on  anything  but  a  unified  philosophy. 
More  likely  training  programs  are  based  on  tradition,  convenience,  cost-containment, 
and  the  whims  of  dozen  or  more  training  directors,  fleet  managers,  and  newly  minted 
AQP  specialists. 


PHILOSOPHY 

POLICIES 

PROCEDURES 

PRACTICES 

Table  11-1.  Degani  and  Wiener’s  “Four  P’s.” 


We  end  this  chapter  on  a  happy  note,  an  ASRS  report  where  the  pilot  claims  to  have 
saved  the  day  and  gives  credit  to  his  “advanced  maneuvering”  training. 

NARRATIVE:  Conducting  a  visual  approach  to  runway  23L  sidestep  runway  23R  at 
MEX.  At  about  3000  ft  AGL  aircraft  encountered  unexpected  wake  vortex.  Aircraft 
rolled  rapidly  to  right  approximately  45-  55  degrees.  Recovery  initiated  in  accordance 
with  company  training  for  advanced  maneuvers.  2  minutes  later  we  were  told  of  a  heavy 
Airbus  landing  on  runway  23L.  Had  Mexican  ATC  warned  of  the  Heavy,  I  could  have 
flown  above  the  glideslope.  Callback  conversation  with  reporter  revealed  the  following: 
Report  was  used  for  structured  callback  and  following  information  was  obtained. 
Reporter  had  just  completed  the  new  program  initiated  by  his  company  for  advanced 
maneuvers  training  The  experience  with  the  Airbus  was  almost  identical  to  the  simulator 
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training  he  had  completed  He  said  it  was  almost  like  a  time  warp  where  for  a  nano 
second  he  felt  he  was  back  in  the  simulator.  He  feels  that  is  why  he  handled  the  situation 
so  well  and  with  very  little  stress.  He  feels  strongly  that  all  air  carriers  should  institute 
such  a  program  ACCESSION  NUMBER.  307029 
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III.  STUDY  METHODOLOGY 


A.  BASIC  QUESTIONS  AND  PREMISES 


Our  basic  premise  in  designing  this  study  was  that  information  on  the  detailed  features, 
as  well  as  an  assessment  of  the  quality  of  the  training  program,  could  be  obtained  by 
seeking  data  directly  from  the  pilots  involved.  For  example,  information  could  be 
gained  from  questionnaires  (attitudes,  experiences,  etc.),  interviews,  flight-deck 
observations,  and  direct  observation  of  the  ground  school  training.  All  of  these  were 
essentially  subjective  measures;  we  would  like  to  have  more  objective  measures,  but 
these  do  not  exist,  or  cannot  be  obtained  at  a  reasonable  cost,  in  most  training 
programs.  Even  instructors’  evaluations  of  maneuvers,  or  overall  simulator 
performance,  are  essentially  subjective.  The  fundamental  information  upon  which  this 
report  is  based  comes  from  pilot  responses  to  questions  (interviews)  and  questionnaire 
data,  as  well  as  the  authors’  observations.  For  a  brief  review  of  the  attitude  surveys 
related  to  cockpit  automation,  see  Wiener,  1993a.  The  list  of  studies  has  grown  since 
that  writing. 

Longitudinal  Studies 


This  study  is  essentially  anthropological.  We  did  not  manufacture  conditions  or 
manipulate  independent  variables,  as  the  experimenters  did  in  a  previous  automation 
study  (Wiener  et  al.,  1991 ).  Like  anthropologists,  we  accepted  the  “village"  of  transition 
training  as  we  found  it,  attempted  to  learn  something  about  the  culture,  and  sought  to 
be  as  unobtrusive  as  possible.  With  the  exception  of  our  interviews  and  three 
questionnaires,  we  generally  achieved  unobtrusiveness. 

This  experiment  was  designed  as  a  longitudinal  study.  A  longitudinal  study  is  simply 
one  in  which  two  or  more  sets  of  measurements  are  taken  over  the  same  sample  during 
the  span  of  the  experiment.  This  allows  the  analysis  to  include  not  only  absolute 
levels,  but  to  evaluate  change,  or  the  effect  of  time,  or  other  interventions.  This  is  to  be 
contrasted  with  a  cross-sectional  design,  in  which  each  sample  is  observed  only  once. 

The  advantage  of  a  longitudinal  study  is  its  sensitivity  to  change.  In  this  and  many 
other  human  factors  studies  of  cockpit  automation,  the  primary  tool  is  the  attitude  scale. 
A  more  objective  dependent  variable,  one  that  is  a  sensitive  measure  of  the  strength 
and  weaknesses  of  automation,  is  desirable.  Such  measures  are  seldom  available. 
Even  in  costly  and  time-consuming  simulator  studies,  there  are  seldom  any  objective 
dependent  variables  to  measure,  and  the  experimenter  again  turns  to  subjective 
measures  of  automation  effects.  For  an  example,  see  Wiener,  et  al.,  1991 .  In  that 
study,  an  attempt  to  measure  the  effect  of  automation  on  the  communication  of  crews 
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flying  a  LOFT  scenario,  the  independent  variable  of  automation  was  achieved  by 
collecting  data  from  crews  flying  the  same  LOFT  in  two  models  of  the  same  aircraft:  the 
traditionally  configured  DC-9-30,  and  its  high-tech  (glass  cockpit)  derivative,  the  MD- 
88.  Even  here,  with  full  simulation,  and  a  highly  scripted  LOFT,  we  ultimately  had  to 
rely  on  observers  and  simulator  instructors  to  provide  the  raw  data  of  the  analysis.  For 
a  thorough  discussion  of  this  question,  we  recommend  Gregorich  and  Wilhelm  (1993) 

While  sensitivity  to  change  over  time  is  the  strong  point  of  longitudinal  studies,  there 
are  also  weak  points  and  disadvantages.  The  more  serious  disadvantages  are  cost, 
and  loss  of  subjects  from  the  original  sample.  The  experiments  are  costly  because  they 
must  be  continued  over  time  in  order  to  obtain  two  or  more  data  collection  points  (which 
we  will  refer  to  a  “phases”  in  this  study).  In  this  experiment  there  were  three  phases. 
The  greatest  hazard  in  longitudinal  analysis  is  the  steady  and  unavoidable  loss  of 
subjects.  Subjects  lose  interest  and  drop  out,  simply  do  not  fill  out  questionnaires,  or 
fail  to  appear  for  interviews  for  a  variety  of  reasons.  They  may  retire,  die,  or  be 
reassigned,  become  medically  disqualified,  or  most  likely,  change  address  and  fail  to 
notify  the  experimenters. 

In  some  experiments  (such  as  this),  subjects  who  drop  out  for  reasons  of  reassignment 
may  be  of  particular  value.  In  the  present  experiment,  these  were  pilots  who  at  least 
completed  the  B-757  transition  program,  and  then  were  reassigned  to  other  aircraft,  for 
administrative  reasons  related  to  new  aircraft  delivery.  We  were  eager  to  speak  with 
these  pilots  because  of  the  shortage  of  information  on  “reverse  transition”,  going  from  a 
modern  aircraft  back  to  a  traditional  cockpit  (see  previous  chapter,  pp.  15-16,  for  a 
discussion  of  “backward  transitioning”). 

The  effect  of  a  pilot  receiving  training  and  possibly  line  exposure  to  the  glass  cockpit 
and  then  returning  to  the  traditional  models  is  worthy  of  study:  it  happens  every  day, 
and  we  know  little  about  it  (Wiener,  1989).  Questionnaires  2  and  3  (Q2  and  Q3) 
contained  a  question  specifically  for  pilots  who  had  made  a  reverse  transition  (see 
Appendix  D).  Some  pilots  in  the  early  days  of  the  757  program  were  sent  back  to  their 
former  planes  to  await  available  seats  in  the  757.  As  aircraft  deliveries  accelerated, 
pilots  quickly  moved  back  to  the  757  line. 

The  human  subject  in  a  long-term  experiment  is  not  an  inanimate  object  or  a  lab 
animal.  There  are  inevitable  changes  in  his/her  existence  that,  quite  apart  from  aircraft 
training  and  line  experience,  affect  a  pilot’s  lifestyle,  flying  habits,  and  certainly  on-the- 
job  attitudes.  The  volunteer  in  a  longitudinal  study,  particularly  a  pilot  in  a  highly 
dynamic  industry,  is  a  moving  target. 
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Organization  of  This  Report 


Chapters  in  this  report  are  designated  by  Roman  numerals.  Each  chapter  has  its  own 
page  numbering,  in  Arabic  numerals.  Thus  page  V-7  is  the  seventh  page  of  Chapter  5. 
The  figures  and  tables  are  likewise  numbered.  Some  of  the  graphics  will  not  be 
assigned  figure  numbers,  since  they  already  have  internal  numbers  as  part  of  the  label. 

The  appendices  are  in  Chapter  XII,  and  are  designated  alphabetically,  again  each  with 
its  own  page  number  (e.g.  D-4).  References  are  based  on  authors’  names,  and  year  of 
publication  (e.g.,  "Sherman,  1997")  and  are  listed  in  Chapter  X. 

Basic  Design  of  the  Study 

This  study  was  designed  as  a  three-phase  longitudinal  experiment.  The  phases  are 
data  collection  points,  in  time,  which  we  designate  as  “PI",  “P2",  and  “P3".  The  three 
phases  are  based  on  the  pilot  volunteer’s  entry  into  the  program,  as  follows: 

PI :  The  first  day  of  transition  class  (ground  school). 

P2:  Approximately  3-4  months  later,  a  time  at  which  a  pilot  will  have  completed 

transition  training,  including  ground  school,  simulator,  and  initial  operating 
experience,  and  will  have  started  line  flying.  If  there  were  no  delays  in 
assignment  to  the  757,  the  pilot  should  have  a  month  or  two  of  line  flying 
following  IOE  before  receiving  his  P2  questionnaire  form  (Q2). 

P3:  Approximately  12-14  months  after  P2,  when  the  pilot  would  have  about  700-900 

hours  of  line  experience  in  the  757. 

The  location  in  time  of  PI  was  fixed:  the  first  day  of  transition  training.  A  package 
consisting  of  the  first  questionnaire  (Q1 )  was  distributed  by  the  757  fleet  manager  at 
the  beginning  of  the  first  day  of  ground  school.  He  encouraged  the  pilots  to  sign  up. 
The  package  included  the  first  questionnaire,  instructions  on  howto  sign  up,  and  a 
sign-up  sheet  with  informed  consent  form,  and  a  description  of  the  confidentiality 
protection.  The  questionnaire  can  be  found  in  Appendix  D.  Most  pilots  who  signed  up 
did  so  on  the  spot,  filled  out  the  questionnaire  and  mailed  it  to  the  authors  at  NASA 
Ames.  The  confidentiality  system  is  discussed  below.  No  effort  was  made  to  contact 
those  who  did  not  join  the  study  at  PI . 

1 50  pilots  signed  up  for  the  study  using  the  blank  in  Q1 .  Three  who  signed  up  were 
later  dropped  due  to  incomplete  data.  Pilot  volunteers  were  considered  members  of 
the  cohort  when  their  Q1  arrived  at  NASA  Ames.  They  were  mailed  Q2  approximately 
four  months  later,  and  Q3  approximately  12-14  months  after  that.  Those  who  did  not 
respond  to  Q2  were  still  sent  the  Q3  form. 
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It  is  possible  that  a  pilot  could  fill  out  the  forms  in  the  wrong  order.  That  is,  he  could 
store  Q2  and  not  send  it  in  until  after  Q3  had  been  filled  out.  Although  there  was 
considerable  delay  in  some  of  the  Q2  questionnaires  arriving,  we  have  no  reason  to 
believe  that  any  were  responded  to  out  of  order. 


Figure  111-1.  Participation  (number  of  questionnaires  completed) 
by  phases  of  the  experiment. 


Thus  the  delivery  of  Q2  and  Q3  was  individually  timed  to  each  pilot’s  entry  into  the 
program.  However,  the  time  at  which  the  form  was  filled  out  and  mailed  back  to  us  at 
Ames  was  completely  under  the  pilot’s  control. 

In  summary,  the  basic  experiment  consisted  of  three  applications  of  questionnaires, 
including  among  other  questions,  an  attitude  scale  of  20  to  24  items  (“probes”).  A 
perfect  record  would  be  a  pilot  volunteer  filling  out  all  three,  and  returning  them  in  a 
timely  manner  for  inclusion  in  the  database. 

Training  Books:  Data  Discontinued 

The  most  objective  data  that  we  had  available  were  grades  as  recorded  in  what 
Continental  calls  training  books.  These  are  essentially  grade  books,  with  pages  in  the 
same  order  as  the  training  syllabus.  At  the  end  of  each  graded  session  (e.g.  simulator 
periods)  the  instructor  would  enter  the  grade.  The  pilot  trainee  retained  his  own  book 
as  he  progressed  through  the  program.  When  the  training  was  completed,  the 
necessary  sign-offs  were  made  in  the  book.  When  everything  was  complete  and 
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satisfactory,  the  pilot  delivered  the  training  book  to  his  chief  pilot,  and  at  this  point  the 
pilot  was  ready  for  initial  operating  experience  (IOE)  and  line  qualification  in  the 
airplane.  The  training  book  was  destroyed.  No  grades  were  kept  --  only  the  fact  that 
the  training  was  completed. 

The  training  books  were  made  available  to  the  project.  Unfortunately  they  proved  of 
little  value,  since  nearly  all  of  the  grades  recorded  were  the  same.  This  was  due  to  the 
fact  that  in  airline  training,  the  company  is  looking  for  standardization  and  conformity  to 
a  standard.  It  is  not  looking  for  unusual  virtuosity.  This  is  somewhat  embedded  in  the 
nature  of  airline  pilot  training.  A  maneuver  can  be  done  satisfactorily  or 
unsatisfactorily;  it  cannot  be  done  “beautifully.”  If  some  grades  were  unsatisfactory,  the 
maneuver  was  repeated.  Given  the  skill  and  motivation  of  the  trainees,  and  the 
instructional  skills  of  the  trainers,  it  is  not  surprising  that  nearly  every  grade  was 
“satisfactory.”  Therefore,  to  our  disappointment,  the  training  books  were  of  no  value  for 
statistical  evaluation,  and  were  destroyed. 


CAL’s  grading  system 

1  Close  to  perfect 

2  Excellent 

3  Average 

4  Satisfactory,  but  needed  to  be  repeated 

5  Unsatisfactory 


B.  QUESTIONNAIRES:  ATTITUDE  SCALES, 
DEMOGRAPHY,  AND  FLYING  EXPERIENCE 


Questionnaires 

The  questionnaires  are  included  in  Appendix  D.  Q1  and  Q2  are  reproduced  in  toto\  Q3 
is  essentially  the  same  as  Q2,  except  for  some  demographic  questions,  which  can  be 
found  on  page  D-10  To  conserve  space,  the  parts  of  Q3  common  to  Q2  were  not 
replicated  in  this  report. 

The  questionnaires  consisted  of  three  parts: 

1 .  A  Likert-type  attitude  scale  (see  example  below)  dealing  with  opinions  about 
flight  safety,  piloting,  and  particularly  cockpit  automation.  There  are  20  items 
(“probes”)  in  Q1  and  24  in  Q2  and  Q3.  Certain  items  in  Q2  and  Q3  were 
inappropriate  for  Q1,  since  the  pilots  had  not  yet  taken  their  757  training. 
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2.  Demographic  data,  mainly  questions  about  past  flying  experience,  but  also 
questions  about  age,  computer  usage,  and  aircraft  preferences. 

3.  Open-ended  questions  which  allowed  pilot  volunteers  to  express  their  beliefs 
and  feelings  in  their  own  words.  These  were  read  and  classified  somewhat 
subjectively  by  the  experimenters  (see  Chapter  VI).  No  statistical  treatment  was 
performed  on  these  data. 

4.  Q1  was  both  a  recruiting  and  data  gathering  instrument.  It  contained  a 
description  of  the  study  and  a  sign-up  sheet,  and  an  informed  consent  form.  In 
addition  it  contained  a  20-item  Likert  scale,  in  order  to  measure  pilots’  attitudes 
as  they  entered  757  training.  Demographic  information  was  also  obtained  (see 
Chapter  IV). 

With  the  loss  of  the  anticipated  data  from  the  training  books,  we  had  to  rely  more 
heavily  on  the  questionnaires,  interviews,  and  direct  observations.  The  raw  data  from 
the  questionnaires  were  entered  into  a  computer  database  at  NASA  Ames,  and  data 
files  were  sent  to  the  University  of  Miami  for  analysis.  Statistical  analysis  and  graphics 
design  were  performed  using  the  SPSS  for  Windows  6. 1(a)  ™  package.  Most  of  the 
demographic  data  are  displayed  graphically  in  Chapter  IV. 

Likert  Scales 


Likert  scales  for  measurement  of  attitude  are  in  wide  use.  They  are  easy  to  design, 
easy  to  administer,  and  the  format  is  generally  familiar  to  the  population  being  sampled. 
In  brief,  an  item  consists  of  a  statement  (“probe")  which  can  be  positively  or  negatively 
stated.  The  respondent  replies  by  accepting  one  choice  of  a  multiple  choice  of  items 
showing  the  agreement/disagreement  with  the  probe,  and  the  degree  of  this  sentiment. 
This  is  called  an  intensity  scale:  the  respondent  states  not  only  whether  he/she  agrees 
or  disagrees,  but  the  intensity  of  this  belief.  Usually  there  is  an  odd-number  of  choices, 
and  the  center  is  one  of  neutrality.  The  center  choice  is  somewhat  ambiguous:  it  could 
possibly  mean  “no  opinion”,  “undecided”,  “don’t  care”,  or  a  truly  neutral  position  on  the 
content  of  the  probe. 

For  a  summary  of  the  results  of  several  studies  employing  this  technique  to  measure 
attitudes  toward  cockpit  automation,  see  Wiener  (1993,  pp.  216-220).  Since  the 
publication  of  the  first  review  there  have  been  more  such  studies,  including  a  large- 
scale  sampling  of  U.S.  air  carrier  pilots  by  Sherman  (1997). 

Likert  data  can  be  treated  as  coming  from  a  nominal  scale  (“strongly  agree”  and 
“agree”  are  simply  categories  of  response,  having  no  ordinal  or  numerical  relationship 
to  each  other,  or  as  an  ordinal  scale,  meaning  that  the  responses  to  the  probe  could  be 
put  in  a  logical  order:  “strongly  agree”  is  stronger  endorsement  of  the  probe  than 


111-6 


“agree”,  which  is  stronger  than  “neutral”,  etc.  Many  experimenters  treat  Likert 
responses  as  if  they  are  from  an  interval  scale,  attaching  numerical  values  to  the 
responses.  For  example,  “strongly  agree”  would  be  scored  as  a  “1”,  “agree”  as  a  “2", 
etc.,  and  the  results  handling  statistically  as  if  interval  scores  had  been  generated.  The 
problem  here  is  that  the  numbers  and  intervals  are  entirely  arbitrary:  using  1,2, 3, 4, 5  as 
numerical  values  treats  the  distance  between  responses  as  psychometrically  equal:  the 
distance  in  attitude  intensity  between  “strongly  agree"  and  “agree”  would  be  assumed 
to  be  the  same  as  between  “agree”  and  “neutral,”  a  questionable,  though  oft-made, 
assumption. 

The  Likert  data  are  displayed  as  in  the  example  below  at  various  places  in  this  report 
The  entire  set  of  graphics  are  displayed  three  to  a  page  (Q1 ,  Q2,  and  Q3)  so  that  the 
reader  can  view  longitudinal  differences.  The  graphs  are  found  in  Appendix  A. 

Demographic  and  Flying  Experience  Data 

All  three  questionnaires  contained  questions  of  a  demographic  nature;  most  dealt  with 
flying  experience,  at  Continental  and  elsewhere.  Most  of  these  data  are  displayed 
graphically  in  the  following  chapter.  Due  to  the  attrition  in  the  study,  the  sample  sizes 
vary  as  shown  previously.  The  demographic  data  are  based  on  all  of  the 
questionnaires  that  we  received.  Thus  some  pilots  may  appear  in  Q2  or  Q3  or  both,  in 
these  displays.  All  pilots  appear  in  the  Q1  data  --  filling  out  that  form  was  the  entry 
path  into  the  study. 


Figure  111-2.  Typical  graphic  representation  of  Likert  scale  results. 


Confidentiality 


Volunteer  pilots  were  assured  of  confidentiality.  A  method  which  was  previously  used 
in  our  studies  was  employed:  for  details,  see  Wiener  (1989,  p.  12).  The  confidentiality 
system  was  based  on  self-assigned  combinations  of  letters  and  numerals  to  a 
maximum  of  six  characters.  In  the  portion  of  the  Q1  form  where  the  pilot  signs  up  to 
join  the  study  (Appendix  D-2),  pilots  were  instructed  to  assign  themselves  a  code  which 
they  could  remember,  but  would  not  identify  them.  They  were  also  given  a  self- 
adhesive  tag  with  a  matrix  of  boxes  into  which  they  could  enter  their  ID  code;  it  was 
suggested  that  they  keep  it  in  a  flight  manual.  The  code  was  attached  to  the  sign-up 
page.  When  we  received  the  Q1 ,  we  set  up  a  separate  computer  file  with  the  ID  code 
and  the  pilot’s  name  and  address,  so  that  we  could  contact  the  pilot  if  need  be.  For 
example,  we  occasionally  received  a  form  with  an  entire  page  inadvertently  left  blank. 
No  other  record  could  link  the  pilot  to  his  code,  and  this  record  has  since  been 
destroyed.  Q2  and  Q3  contained  only  the  ID  code,  no  names  or  addresses. 

Prior  to  recruiting  pilot  volunteers,  we  met  with  the  Safety  Committee  of  the  IACP  to 
outline  the  study  and  discuss  the  confidentiality  plan.  No  concerns  about 
confidentiality  were  raised  by  the  IACP  representatives,  and  they  readily  agreed  to 
support  the  study.  The  investigators  offered  to  brief  IACP  on  the  progress  of  the  study, 
or  to  hold  joint  management-IACP  briefings.  Several  of  these  meetings  have  been 
held,  and  cooperation  with  IACP  was  excellent. 

In  other  contacts  with  the  pilots,  confidentiality  was  also  preserved  by  whatever  means 
necessary.  For  example,  in  the  face-to-face  and  telephone  interviews  (next  section), 
we  could  not  pretend  that  we  did  not  know  whom  we  were  talking  to.  We  simply 
explained  this,  and  assured  the  pilot  that  we  would  not  record  any  names  or 
identification  codes  with  the  interview  notes.  The  pilots  were  satisfied  with  this,  in  fact 
no  question  about  confidentiality  was  ever  raised.  On  jumpseat  observations,  no 
record  was  kept  of  crew  names  or  flight  numbers,  dates,  origins  or  destinations.  We 
feel  safe  in  saying  that  confidentiality  was  simply  not  an  issue  in  this  study. 


C.  OTHER  SOURCES  OF  INFORMATION 


Interviews 


Two  sets  of  interviews  were  conducted.  The  first  was  at  the  initial  period  of  the  study, 
before  757  school  had  begun.  The  interviewees  were  flight  management  personnel, 
beginning  with  the  vice-president  for  flight  operations.  Following  him  were  flight 
standards  pilots,  the  757-767  fleet  manager  and  assistant  fleet  manager,  and  others  in 
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the  management  hierarchy.  These  interviews  were  face-to-face,  and  were  conducted 
mostly  one-on-one,  with  a  few  being  one  on  two. 

The  purpose  of  these  interviews  was  to  determine  management  attitudes  toward 
automation,  training  methods,  and  standardization,  and  what  problems  they  anticipated. 
The  experimenter  asked  prepared  questions,  but  allowed  the  interviewee  ample  room 
to  discuss  anything  he  wished.  The  information  gleaned  from  the  interviews  was  not 
treated  as  data,  but  as  background  material. 

The  interviews  with  the  management  pilots  yielded  the  following  information. 

1 .  A  strong  confidence  in  the  choice  of  the  757-767  and  the  important  role  of  these 
aircraft  in  the  modernization  of  CAL’s  fleet,  and  its  route  plans. 

2.  A  strong  approval  of  the  training  plans  and  syllabi  being  drawn  up  by  the  fleet 
managers  and  their  staff  persons. 

3.  Concern  about  safety  problems  in  highly  automated  aircraft,  and  the  ability  of 
management  to  ensure,  though  training  and  other  support,  that  automation 
would  not  be  a  problem. 

4.  Concern  about  standardization  in  general,  and  the  ability  of  flight  management 
to  standardize  the  757-767  fleet  to  harmonize  with  the  other  fleets  at 
Continental.  The  question  of  cross-fleet  standardization  and  the  desire  not  to 
make  the  automated  aircraft  “odd-balls”  permeated  every  discussion.  [These 
interviews  were  completed  nearly  three  years  before  the  company  made  the 
decision  to  buy  the  fleet  of  Boeing  jets], 

Jumpseat  Observations 

The  three  authors,  and  one  graduate  student  assigned  to  the  project,  made  a  number 
of  jumpseat  observations  of  line  trips  on  the  757.  This  was  for  familiarization;  no  data 
were  collected  on  these  flights. 

Ground  Schools 


Two  of  the  authors  attended  ground  school  on  the  757,  including  the  program  on 
Human-Centered  Automation  Training  (H-CAT),  and  the  international  qualification 
class.  Both  authors  had  a  CBI  access  code  and  worked  on  this  instruction  while  in 
ground  school. 
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Standardization  Meetings 


Two  of  the  authors  attended  some  of  the  757  standardization  committee  meetings  for 
instructional  purposes,  and  at  times  were  called  on  for  advice  on  matters  of  checklists 
and  procedures.  The  study  team  made  several  presentations  on  the  progress  of  the 
study  in  standardization  meetings 
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IV.  CHARACTERISTICS  OF  THE  VOLUNTEER  COHORT 


A.  OVERVIEW 


In  this  chapter  we  provide  in  graphic  and  tabular  form  certain  demographic  information 
provided  about  themselves  by  the  pilot  volunteers.  Most  of  the  information  deals  with 
flying  time  and  experience  in  various  cockpits;  a  minor  portion  of  the  chapter  covers 
such  variables  as  the  volunteers’  age,  computer  usage,  and  preferences  for  aircraft. 

The  chapter  is  organized  along  the  three  longitudinal  phases  of  the  study  and  the 
questionnaire  data  collected  at  each  phase. 

Representativeness  of  the  Sample 

As  we  noted  in  a  previous  NASA  report  (Wiener,  et  al.,  1991 ),  an  experimenter  can 
never  be  certain  that  the  sample  of  volunteers  is  truly  representative  of  the  overall 
population  (all  pilots  in  some  circumscribed  group).  We  asked  ourselves  the  following 
question:  are  people  who  volunteer  for  a  project,  who  are  willing  to  give  their  time  for  no 
direct  gain,  attitudinally  different  from  those  who  do  not  respond  to  the  call  for 
volunteers?  This  problem  is  known  in  statistical  sampling  as  “non-response  bias.” 

We  have  no  ready  answer  to  this  question  -  the  possibility  of  non-response  bias 
plagues  any  study  based  on  volunteers.  It  is  generally  supposed,  but  seldom  proven, 
that  volunteers  for  a  study  such  as  this  may  be  the  “sharper”,  more  capable,  more 
motivated  end  of  the  continuum  of  aptitudes  among  the  population  from  which  they  are 
drawn.  If  this  is  true,  it  may  be  extended  to  assume  that  the  attitudes  expressed  are 
more  positive  toward  fleet  modernization  and  the  role  of  automation.  It  might  follow 
from  this  that  the  sample  would  contain  proportionally  more  young  pilots  than  the 
population,  but  this  is  speculation  also. 

Why  does  a  pilot  volunteer  to  serve  in  such  a  study?  We  feel  that  we  know  the  answer 
to  this  from  interviews  and  open-ended  questions.  First,  many  are  curious  about  the 
study,  and  many  feel  that  volunteering  for  a  study  is  the  professional  thing  to  do. 

Others  may  be  attracted  by  the  technological  reputation  and  mystique  of  NASA. 

Finally,  we  feel  that  many,  perhaps  most,  of  the  volunteers  were  drawn  to  the  study  by 
the  persuasion  of  a  popular  fleet  manager  who  personally  called  for  volunteers  at  the 
first  meeting  of  each  new  transition  training  class. 

We  have  not  answered  the  question  of  non-response  bias.  We  have  found  no  obvious 
bias  in  our  sample.  We  have  every  reason  to  believe  that  these  responses  can  be 
generalized  to  the  population  of  Continental  pilots  who  bid  the  757  in  those  years.  This 
may  present  a  bias  in  itself:  who  among  the  CAL  pilots  leaves  a  comfortable  position  in 
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a  traditional  aircraft,  and  for  no  monetary  gain  (see  below)  accepts  the  trouble  and  the 
risk  of  a  transition  to  a  modern  aircraft?  We  have  some  answers  to  that  question:  in 
previous  studies  we  asked  volunteers  why  they  bid  the  glass  aircraft.  The  answer  was 
usually  framed  in  terms  of  seeing  glass  as  the  future  of  commercial  aviation,  and  their 
desire  to  be  on  the  leading  edge  of  that  trend.  A  number  of  the  attitude  probes  (see 
page  A-4)  are  aimed  at  pilots’  motivation  to  bid  the  new  aircraft. 

Migration  Patterns 

At  the  time  of  our  study,  Continental,  unlike  nearly  every  other  airline  in  the  U.S.,  did 
not  pay  pilots  according  to  a  “weight  differential."  Pay  was  based  on  longevity  at  the 
company,  and  not  on  the  weight  of  the  aircraft  flown.  Longevity  affected  bidding  for 
aircraft  seats,  trips,  and  bases,  as  it  does  at  other  airlines.  This  fact  has  implications 
throughout  the  company,  as  well  as  for  this  study.  One  of  the  things  that  the  non- 
differential  system  does  is  to  discourage  “seat  hopping,”  a  practice  which  is  costly  for  a 
training  department,  and  generally  wasteful  of  pilot  talent.  In  other  airlines,  pilots  bid 
from  plane  to  plane  in  order  to  move  up  the  path  of  weight  differential  and  higher  pay. 
Most  pilot  contracts  attempt  to  limit  seat  hopping,  due  to  its  impact  on  training  cost  and 
flight  proficiency.  Some  pilots  seat  hop  in  order  to  collect  type  ratings,  as  a  hedge 
against  possible  job  loss  and  re-entry  into  the  market. 

At  Continental,  pilots  tend  to  move  around  until  they  find  a  plane,  a  base,  and  a 
schedule  that  serves  their  needs,  and  stay  put  for  what  seems,  by  standards  of  other 
airlines,  a  long  stand.  The  recent  base  closings  and  reductions,  with  Newark  and 
Houston  expanding,  has  shattered  some  pilots’  plans  which  were  based  on  location 
rather  than  aircraft.  With  no  weight  differential,  pilots  based  their  bids  on  their  own 
convenience,  and  did  not  particularly  care  which  aircraft  they  flew.  Bids  reflected 
desires  for  bases,  schedules,  long  periods  off,  etc.,  and  not  so  much  for  aircraft  type. 

Non-differential  pay  schedules,  though  financially  efficient  for  the  company,  can  lead  to 
some  peculiar  results,  e.g.  senior  captains  flying  low-end  aircraft  (DC-9,  B-737-200), 
while  junior  captains  were  flying  DC-1  Os  over  the  Pacific.  We  once  rode  jumpseat  in  a 
DC-9  with  a  captain  who  was  one  of  the  most  senior  in  the  airline.  With  his  seniority, 
why  was  he  flying  a  DC-9,  when  “heavy  metal"  (DC-1  Os  and  B-747s)  and  international 
schedules  were  available  to  him?  His  reply  was  that  he  was  tired  of  hotel  living,  and 
with  his  seniority  could  consistently  bid  out-and-back  trips  from  Houston  (IAH).  He 
boasted  that  in  the  last  five  years  he  had  spent  only  one  or  two  nights  a  month  in  a 
hotel. 

We  found  in  our  early  interviews  that  one  of  the  motivators  for  a  757  bid  was  the 
anticipated  fleet  of  767s,  which  was  on  order  for  ETOPS  operations  across  the  North 
Atlantic,  a  highly  desirable  route.  This  turned  out  to  have  its  negative  side  for  this 
study.  When,  for  economic  reasons,  the  company  decided  to  cancel  its  767  order, 
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there  was  widespread  dissatisfaction  among  the  757  pilots,  due  to  both  the  reduction  of 
the  ETOPs  flights,  and  the  757  schedule  in  general.  At  Continental  the  757  and  767 
are  in  the  same  fleet.  Because  the  company  could  not  make  good  on  the  767  lines, 
they  agreed  to  waive  the  two-year  rule  (training  freeze)  and  allow  the  crews  to  bid  off  of 
the  757.  This  resulted  in  a  serious  loss  of  pilots  in  our  cohort,  one  of  the  hazards  of 
longitudinal  studies,  as  noted  previously.  The  757  program  proceeded,  and  a  new  fleet 
of  777s  and  767s  are  now  being  delivered. 

Thus,  a  combination  of  the  loss  of  the  767  fleet,  the  generally  undesirable  schedules 
(in  terms  of  the  effort  required  to  earn  flight  hours),  and  the  assignment  of  more  and 
more  757  time  to  Newark  (EWR)  and  less  to  IAH  led  to  heavy  out-migration  from  the 
program  in  the  first  year  of  our  field  study.  757  time  scheduled  at  the  Guam  base 
remained  unchanged.  The  situation  has  now  stabilized,  and  the  promise  of  the  767s, 
with  first  deliveries  scheduled  for  1999,  has  once  again  made  this  a  desirable  fleet. 
[Deliveries  of  26  767-400s  will  begin  in  July  2000], 

In  June  1998,  after  data  collection  on  this  study  had  been  completed,  a  new  contract 
changed  all  of  this,  and  put  Continental  pilots  on  a  traditional  seniority  and  weight 
differential  basis.  The  entire  airline  had  a  “flush  bid",  meaning  that  every  pilot  could  bid 
any  plane  in  the  fleet.  The  bidding  at  CAL  was  simplified  somewhat  by  a  contract  that 
created  only  three  weight  classes: 

Narrow  body  DC-9,  737-300/500/700 

Mid-body  727,  MD-80,  737-800,  757 

Wide  body  DC-1 0,  767,  777 


B.  PHASE  1  DATA 


Repeating  what  was  said  previously,  the  first  questionnaire  (Q1 )  was  attached  to  the 
invitation  to  join  the  study,  To  sign  up,  the  pilot  filled  out  the  questionnaire,  which 
included  the  questions  whose  results  are  presented  in  this  chapter,  as  well  as  a  20-item 
Likert  attitude  scale.  There  was  one  page  that  dealt  with  the  confidentiality  and  self¬ 
assignment  of  the  ID  code,  and  an  informed  consent  sign  off.  The  completed  forms 
were  then  sent  to  us  at  NASA  Ames  Research  Center  in  an  addressed,  stamped 
envelope  provided  with  the  invitation  and  questionnaire. 

One  attitude  probe  which  appeared  on  Q1  that  was  not  appropriate  in  the  later  phases: 
No.  8a,  “I  am  very  apprehensive  about  going  through  this  transition”  (see  Figure  111-2, 
previous  chapter).  The  results  of  the  Likert  probes  are  shown  in  Appendix  A 
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Miscellaneous  Information 


Of  the  101  volunteer  pilots  who  submitted  all  three  questionnaires,  all  but  one  were 
males.  One  other  female  pilot  filled  out  Q1 ,  but  we  never  received  data  from  her  again. 
Accordingly,  we  use  principally  the  male  gender  in  this  report.  Volunteers  were  asked 
to  give  their  age  to  the  closest  month.  We  converted  this  to  years  for  graphic  purposes 
(see  bar  chart  below).  The  age  distribution  is  consistent  with  what  we  have  seen  in 
other  field  studies:  the  757  tends  to  be  a  mid-career  choice.  At  other  airlines  with 
weight  differentials  in  their  contracts,  bidding  the  757  represented  something  of  a 
stepping  stone,  from  lighter  (and  therefore  less  lucrative)  aircraft  such  as  the  B-737  and 
DC-9/MD-80  to  the  “heavy  metal.”  At  CAL,  with  no  weight  differential,  and  various 
reasons  for  bidding  the  757,  we  still  see  mid-career  pilots  making  this  transition. 


FIGURE  IV-1.  Age  of  pilot  volunteers. 


We  were  astonished  at  the  number  of  “older”  (with  respect  to  a  mandatory  retirement  at 
age  60)  pilots  bidding  the  757(see  Figure  IV-1  above).  About  1/3  of  the  pilots  were 
within  ten  years  of  retirement  when  they  filled  out  Q1.  With  nothing  to  gain  monetarily, 
this  bid  probably  represented  a  desire  to  fly  a  modern  aircraft  before  retirement.  Many 
reported  to  us  that  even  with  only  five  to  ten  years  left,  they  saw  the  company  rapidly 
expanding  with  a  glass  fleet,  and  wanted  to  be  part  of  that  movement.  We  will  look  y  at 
where  (what  aircraft)  these  pilots  arrived  from.  Professional  pride  played  a  big  part  in 
the  bidding. 

We  have  said  little  so  far  about  first  officers.  Much  of  what  we  have  already  said  of 
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captains  applies  as  well  to  the  first  officers,  and  as  we  will  see,  their  attitudes  are 
remarkably  similar.  Our  first  officers  tended  to  be  mid-career  in  that  seat,  and 
jockeying  for  position  as  a  captain  of  a  smaller  aircraft.  Some  senior  first  officers 
remained  in  the  right  seat  of  the  757,  awaiting  an  opportunity  to  bid  757  captain.  For 
most  first  officers,  unless  they  are  very  senior,  a  more  inviting  route  is  to  bid  captain  in 
the  expanding  fleet  of  B-737-500s  and  next  generation  737s.  Here  they  will  find  the 
opportunity  to  utilize  glass  cockpit  knowledge  and  skills  learned  on  the  757.  As 
seniority  builds,  there  is  always  a  future  in  the  expanding  757/767  fleet. 

The  present  base  to  which  the  volunteer  was  assigned,  and  his  expected  post¬ 
transition  base,  were  also  asked  on  Q1.  The  present  base  is  displayed  in  the  pie  chart, 
Figure  IV-2.  The  anticipated  future  bases  are  easily  summarized: 


NO.  % 


Base 


Houston 

Newark 

Other 


64  43.5 

78  53.1 

5  3.4 


Note  that  these  are  pilots’  expectations;  the  reality  of  assignments  is  where  the 
company  chose  to  base  its  757  fleet.  As  it  turned  out,  these  were  fairly  realistic 
estimates  in  the  aggregate.  One  can  easily  see  the  influence  of  Continental’s  two 
primary  bases,  Houston  and  Newark.  During  the  course  of  the  study  there  was 
considerable  base  realignment  and  closing,  including  the  once  powerful  and  highly 
desired  Denver  base.  Presently  757s  are  based  only  at  Houston,  Guam,  and  Newark. 
This  report  essentially  ignores  the  Guam  base,  which  even  now  is  a  small,  somewhat 
remote  part  of  the  757  operation.  The  concentration  of  757s  at  Newark  has  turned 
Continental  757  pilots  into  a  tribe  of  commuters,  many  continuing  to  live  near  formerly 
thriving  bases  such  as  Denver  and  Los  Angeles,  locations  of  reduced  importance  at 
Continental,  but  still  considered  desirable  places  to  live. 
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Present  Base 


FIGURE  IV-2.  Base  of  volunteers  at  time  of  757 
transition. 


Computer  Experience 

In  a  previous  study  (Wiener,  et  al.,  1991 )  the  authors  were  curious  about  the  pilot’s 
computer  experience,  and  hoped  to  relate  this  to  performance  in  the  cockpit. 
Unfortunately  we  asked  the  question  the  wrong  way,  asking  merely  was  there  a 
computer  in  the  home?  We  should  have  asked  who  used  it.  We  sought  to  remedy  that 
problem  in  this  questionnaire  by  asking  if  there  was  a  personal  computer  in  his  home, 
and  if  so,  how  often  did  he  (the  pilot)  use  it?  For  the  exact  format,  see  page  D-4, 
questions  10  and  11. 

The  answer  to  the  first  question  (see  Table  IV-1 ),  “do  you  use  a  personal  computer  in 
your  home,”  yielded  the  following  results: 


Yes 

76% 

No 

23% 

It  is  often  assumed  that  first  officers,  being  younger  than  captains  and  therefore 
educated  in  the  era  of  digital  computers,  are  more  likely  to  be  intrigued  by  and 
competent  in  computers.  One  might  expect  to  find  a  higher  proportion  of  first  officers 
with  home  computers.  This  is  argumentative:  one  could  advance  the  position  that 
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captains  can  more  easily  afford  computers.  First  officers  may  not  have  the  cash  to  put 
into  computers.  We  examined  this  by  casting  the  data  into  a  2-by-2  table  and 
performing  a  chi-square  test  of  row/column  independence.  This  resulted  in  a  chi- 
square  of  1 .03,  (df  =  1 ),  indicating  that  the  difference  in  home  computer  usage  between 
captains  and  first  officers  (80  %  vs.  73  %)  was  not  statistically  significant. 

For  the  roughly  3/4  of  the  sample  that  responded  positively,  the  breakdown  by  level  of 
usage  is  given  in  Figure  IV-3.  Well  over  half  of  the  sample  reported  usage  daily  or 
several  times  weekly.  We  again  examined  computer  usage  by  captains  and  first 
officers,  casting  the  data  into  a  2-by-4  matrix  (Table  IV-la).  Again  the  result  was  again 
non-significant:  chi-square  =  2.04  (df  =  3). 

In  summary  we  find  that  in  our  sample  of  146  pilots,  about  3/4  report  having  and  using 
a  personal  computer,  with  fairly  uniform  distribution  of  cases  over  the  four  levels  of 
usage.  We  find  no  difference  between  captains  and  first  officers  in  the  availability  or 
usage  of  the  home  computer.  We  shall  next  attempt  to  correlate  these  data  with 
attitude  items.  We  are  unable  to  do  what  we  had  originally  planned  (using  the  training 
books),  attempt  to  correlate  computer  usage  with  proficiency  measures. 
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Personal  Computer  Use 


FIGURE  IV-3.  Frequency  of  usage  of  home  computer. 


DO  YOU  USE  A  PC  AT  HOME? 
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67 

17 

84 

45 

17 

62 
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112 

34 

146 

TABLE  IV-1.  Computer  usage  by  seat. 


PERSONAL  COMPUTER  USAGE  BY  SEAT 
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26 

22 

19 

17 

84 
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15 

19 

11 

17 

62 
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41 

41 

30 

34 

146 

TABLE  IV-la.  Frequency  of  usage  of  home  computer  by  seat  in  757. 


In  Table  IV-la  we  have  displayed  frequency  of  computer  usage  by  the  crew  members 
as  a  2-by-4  contingency  table.  We  will  discuss  the  outcome  of  statistical  tests  on  this 
and  other  tables  in  the  next  chapter. 

The  only  other  experimenter  that  we  are  aware  of  who  has  gather  statistics  on 
computer  usage  in  high  tech  crew  members  is  Orlady  (1991 ),  who  asked  pilots  of  high- 
tech  cockpits  how  many  had  home  computers.  The  group  was  about  evenly  split 
Note  that  if  his  report  is  taken  literally,  he  made  the  same  mistake  as  Wiener  et  al.  did 
in  their  1991  study.  The  proper  question  is  not  ownership,  but  usage.  Orlady  took  it  a 
step  further  and  asked  the  group  that  responded  that  they  did  have  computers  whether 
they  felt  their  computer  experience  made  any  difference  in  transition  to  glass.  The 
group  was  about  evenly  split  (Orlady,  1991,  p.  2.12). 
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Choice  of  Aircraft 


We  asked  a  question  we  have  asked  before,  requiring  the  volunteer  to  pick  from  his 
company’s  fleet  the  plane  that  he  would  most  want  to  fly,  quality  of  trips  and  pay  being 
equal  (see  page  D-4,  question  14,  for  exact  wording).  The  results  are  displayed  in 
Figure  IV-4.  The  results  indicate  a  strong  loyalty  for  the  757,  accounting  for  nearly  3/4 
of  the  votes.  The  DC-10  and  the  B-747  accounted  for  most  of  the  rest,  the  remaining 
aircraft,  narrow  bodies  with  one  exception,  obtained  few  votes. 


First  Choice  of  Aircraft  Type 
Regardless  of  Money,  Base,  Etc, 


FIGURE  IV-4.  Pilots’  choice  of  aircraft  in  Continental 
fleet. 


Total  Flying  Time 

The  two  bar  charts  showing  total  flying  time  and  flying  time  at  Continental  reinforce 
what  we  have  said  about  the  mid-career  pilot,  those  in  a  range  of  perhaps  10,000- 
16,000  hours.  We  have  made  a  distinction  in  the  two  flying  time  charts  between  total 
time  and  time  at  Continental.  These  disparities  exist  because,  as  explained  early  in 
this  report,  Continental  is  not  a  “pure  bred”  airline,  but  one  composed  of  many 
tributaries  (see  Chapter  I).  A  large  number  of  the  pilots  came  to  Continental  in  recent 
years,  as  a  result  of  the  airline  mergers  and  acquisitions  engineered  during  the  reign  of 
Frank  Lorenzo.  Thus  the  difference  between  the  hours  of  flying  time  depicted  in  the 
two  figures  (IV-5  and  IV-6)  is  considerable. 
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FIGURE  IV-5.  Total  flying  time,  all  sources. 


FIGURE  IV-6.  Total  flying  time  at  Continental. 
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The  typical  Continental  pilot  in  the  757  program,  at  the  time  we  collected  data  (mid- 
1990s)  had  about  8,000  to  13,000  total  hours,  including  pre-merger  companies, 
military,  general  aviation,  etc.,  about  half  of  which  is  with  Continental.  Like  the  pilots 
whom  we  have  studied  in  other  projects  (Wiener,  1989;  Wiener  et  al.,  1991),  the  B-727 
predominates.  At  every  airline  we  have  studied,  this  is  the  case.  We  call  the  727  the 
“prep  school  for  757.”  Table  IV-2  illustrates  the  importance  of  the  727  in  the  migration 
patterns  of  the  757  pilot. 

Transition  from  the  727  (or  the  DC-9  for  that  matter)  to  the  757  is  a  turning  point  in  a 
pilot’s  career:  a  sweeping  technological  change,  and  a  challenging  training  program. 

At  most  of  the  airlines  we  have  studied,  we  have  encountered  the  “25-year  727  pilot." 
Every  airline  has  a  collection  of  them.  He  (or  she)  has  spent  an  entire  career  in  the 
three  seats  of  the  727,  and  has  little  interest  in  moving.  What  it  takes  is  a  new 
technology  aircraft,  not  just  a  heavier  one,  and  perhaps  a  subtle  threat  that  the  727  is 
going  to  soon  be  retired.  One  thing  that  makes  the  25-year  727  pilot  somewhat 
apprehensive  about  bidding  the  757  is  that  he  has  been  to  school  so  little  during  his 
career  on  the  727,  compared  to  pilots  who  have  migrated  all  over  the  fleet.  And  at 
Continental,  with  no  weight  differential,  why  bother? 

Previous  Cockpit  Positions 

Pilots  were  asked  to  fill  in  a  matrix  similar  to  Table  IV-2,  simply  checking  each  cockpit 
position  they  had  held  at  Continental.  They  were  instructed  not  to  include  flying  time  in 
each  seat,  only  a  check  that  they  had  at  least  once  held  this  seat  at  Continental.  Some 
interesting  facts  come  from  this  table.  We  again  observe  the  importance  of  the  727  in 
the  migration  of  pilots.  Pilots  came  from  variety  of  seats,  including  captains  who  had 
flown  the  three  wide  body  ("heavy”)  jets  that  were  in  operation  by  Continental  when  the 
study  began  (A-300,  B-747,  DC-10).  The  A-300  has  since  been  retired.  Again  turning 
to  our  discussion  of  the  lack  of  weight  differential  pay  scales  at  Continental,  it  is 
probable  that  such  a  bid  would  not  have  occurred  at  other  U  S.  airlines.  The  757  is  a 
mid-weight  aircraft,  somewhat  heavier  than  the  other  narrow  bodies,  far  lighter  than  the 
wide  bodies.  At  an  airline  with  weight  differentials,  it  would  be  a  significant  financial 
sacrifice  for  a  wide  body  pilot  to  bid  the  757,  whatever  his  motive. 
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Previous  Seats  At  Continental 

Aircraft  Type 

Captain 

First  Officer 

Second 

Officer 

Totals 

DC -9 

25 

38 

— 

63 

MD-80 

15 

17 

— 

32 

737  - 100/200 

9 

6 

— 

15 

737  -  300 

23 

22 

45 

727 

52 

64 

57 

173 

A-300 

21 

9 

2 

32 

707/720 

4 

19 

22 

45 

DC  - 10 

26 

37 

23 

86 

747 

14 

14 

6 

34 

Table  IV-2.  Previous  seats  held  at  Continental  by  pilots  in  the  cohort. 


We  next  asked  the  pilots  what  was  their  last  aircraft  before  embarking  on  their  757 
transition.  Figure  IV-7  shows  the  results.  It  is  noteworthy  that  they  came  from  so  many 
aircraft,  with  a  sizeable  number  coming  from  the  three  wide  bodies.  We  have  not 
attempted  to  scale  these  results  to  the  number  of  aircraft  (and  crews)  in  each  fleet.  The 
DC-10,  A-300,  737,  and  727  are  about  equal  in  their  contribution,  and  the  four  account 
for  almost  3/4  of  the  pilots  in  the  cohort.  At  the  time  the  first  757  classes  were  being 
formed,  the  A-300  was  on  the  way  out  of  the  company’s  fleet;  their  pilots  were 
scrambling  for  the  best  deal  they  could  find.  The  737-300  at  the  time  was  the 
company’s  most  modern  cockpit.  Continental’s  models  had  the  flight  management 
system  (FMS),  but  not  the  glass  cockpit.  This  configuration  is  often  called  the  B-737- 
300-non-EFIS.  At  this  time  the  fleet  of  glass  B-737-500s  began  arriving. 
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Last  Aircraft  Before  757  Transition 


FIGURE  IV-7.  Last  aircraft  flown  before  757  transition. 


FIGURE  IV-8.  Months  in  last  seat  before  757  transition. 
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For  the  737-300  pilots,  familiar  as  they  were  with  FMS  functionality  and  Boeing 
terminology,  this  was  a  relatively  easy  transition. 

In  order  to  determine  the  stability  of  assignment  of  the  pilots  who  bid  the  757,  we  asked 
the  number  of  months  in  the  model  flown  before  transition.  These  data  are  displayed  in 
Figure  IV-8.  Examination  of  the  figure  shows  that  a  sizeable  group  (27  per  cent) 
resided  in  the  left-most  bar  (zero  to  24  months).  Following  them  were  three  roughly 
equal  subgroups  (25-96  months)  accounting  for  about  55  per  cent  of  the  sample.  Four 
small  subgroups  of  those  with  a  large  number  of  years  accounted  for  less  that  20  per 
cent  of  the  total. 

How  many  of  the  transitioning  757  pilots  had  previous  glass  experience  at  Continental 
or  elsewhere?  Pilots  were  asked  merely  to  list  any  glass  cockpit  of  any  type  in  which 
they  had  been  assigned.  Table  IV-3  displays  the  results.  About  3/4  were  737-500,  the 
rest  being  a  scattering  of  military  and  executive  jet  cockpits.  Thus,  if  we  exclude  for  the 
moment  previous  737-500  crews,  it  can  be  said  that  the  vast  majority  in  the  cohort  had 
not  been  exposed  to  glass  prior  to  757  transition. 

Glass  or  not,  we  wished  to  know  the  most  advanced  cockpit  which  the  pilot  had  flown  at 
any  time  during  his  career.  The  exact  meaning  of  “most  advanced”  was  left  to  the  pilots' 
discretion.  These  data  are  shown  in  Table  IV-4.  The  table  indicates  a  wide  variety  of 
airline,  military,  and  executive  jet  cockpits  in  the  experience  of  the  cohort.  There  are 
some  discrepancies  between  these  data  and  the  previous  question  about  glass 
experience,  probably  due  to  misunderstanding  the  question. 
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Previous  Glass  Experience 

Aircraft  Type 

Responses 

F-14-A 

1 

737  -  500 

27 

G  -  IV 

2 

Beechcraft  C  -  90B 

1 

F  - 16 

1 

747  -  400 

1 

EMB-120 

3 

FJ50/LR55 

1 

737-500/MU-300 

1 

Saab  340 

1 

ATR  42 

1 

Total 

40 

Table  IV-3.  Glass  aircraft  flown  by  cohort 
prior  to  757  transition. 


For  example,  we  cannot  explain  the  fact  that  ten  pilots  claimed  to  have  757  experience, 
but  this  is  not  reflected  in  the  previous  question  about  glass  experience.  Also  there  is  a 
minor  discrepancy:  31  pilots  claim  737-500  experience,  but  only  27  listed  the  -500  in 
response  to  the  question  about  past  glass  cockpits.  With  the  rapid  fleet  expansion  at 
Continental,  with  large  orders  from  Boeing  for  757,  767,  777,  737-500,  and  recently  for 
next  generation  737s,  and  the  retirement  of  the  older  model  aircraft,  the  figures  will 
change  dramatically  in  the  next  five  years.  If  these  questions  were  asked  five  years 
from  now,  undoubtedly  most  pilots  would  have  glass  experience,  most  would  list  some 
new  model  of  Boeing  aircraft  as  their  most  advanced  cockpit,  and  they  would  not  be 
going  through  glass  transition  for  the  first  time.  Early  in  the  next  century  Continental’s 
fleet  will  be  all  Boeing  and  all  glass. 
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Most  Advanced  Aircraft 

Aircraft  Type 

Total 

727 

6 

737  -  200 

1 

737  -  300 

17 

737  -  500 

31 

747 

4 

757 

10 

A300 

12 

DC  -10 

20 

MD-80 

21 

Saab  340 

1 

ATR  42 

1 

F  -4 

2 

F  - 15 

1 

F  - 16 

3 

FJ50 

1 

C  -141 

2 

53 -A  Viking 

1 

Westwind  1124 

1 

G  III 

1 

G  IV 

1 

Seven  responses  each  involving  two  aircraft  were  not  included  in  the  table: 

1.  DC-10/A-300  5.757/767 

2.  DC-10/L-1011  6.757/767 

3.  DC-10/C-141  7.  DC-9/727 

4.  DC-10/737-300 

TABLE  IV-4.  The  most  advanced  aircraft  flown  prior  to  757  transition. 
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Recent  Experience  with  Various  Instrument  Approaches 


In  past  field  studies  we  have  asked  the  volunteer  pilots  to  estimate  how  many  times  in 
the  previous  12  months  they  have  flown,  either  as  pilot  flying  (PF)  or  pilot  not-flying 
(PNF),  various  instrument  approaches.  [Currently  at  CAL,  the  PNF  is  called  the  “pilot 
monitoring”  (PM)].  The  results  are  displayed  in  the  next  five  figures.  The  data  must  be 
regarded  as  estimates,  based  on  the  pilots’  memory  of  the  previous  year.  These  data 
have  been  deemed  valuable  in  planning  training  for  the  less-frequently  used 
approaches.  Such  data  may  be  particularly  useful  in  planning  training  syllabi  and 
schedules  for  the  Advanced  Qualification  Program  (AQP). 

The  question  sometimes  arises  during  an  accident  investigation,  when  the  pilots  fly  an 
infrequently  used  approach  and  an  accident  results.  Such  was  the  case  in  the  crash  of 
the  U.S.  Air  Force  B-737  (T-43A)  in  Croatia  (Phillips,  1996a).  It  is  not  unusual  in  these 
cases  to  find  that  the  pilot  has  made  few,  if  any,  of  the  less-often  employed  non¬ 
precision  approaches  in  the  last  12  months. 

The  data  on  autolands  present  a  special  case.  Some  of  the  pilots  would  have  spent 
the  last  year  flying  older  aircraft  not  configured  for  autoland,  or  possibly  configured  but 
not  maintained  for  autoland  (see  Fig.  IV-7).  Be  that  as  it  may,  the  frequency  of 
autolands  in  the  pre-1994  Continental  line  experience  of  the  early  757  cohort  was 
virtually  zero.  Close  to  90  per  cent  of  our  volunteers  reported  no  autolands,  and  the 
remaining  frequencies  are  minimal  (see  Fig.  IV-9).  The  following  Continental  aircraft 
were  equipped  authorized  for  autoland  at  the  time  of  our  study:  B-737-300/500  and  B- 
757.  The  MD-80  was  equipped  but  not  used. 


FIGURE  IV-9.  Frequency  of  autolands  in  previous 
year. 
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FIGURE  IV-10.  Frequency  of  ADF  (NDB)  approaches. 


Figures  IV-10,  1 1 ,  and  12  demonstrate  that  the  frequency  of  various  non-precision 
approaches  is  quite  small.  About  55  per  cent  of  the  pilots  report  no  ADF  (more 
correctly,  NDB)  approaches  during  the  previous  year,  and  the  frequency  is  very  low  for 
the  remaining  pilots.  The  frequency  of  localizer  and  VOR  approaches  is  also  quite  low: 
about  30  per  cent  of  the  pilots  report  having  flown  none  of  these  two  approaches  during 
the  previous  year. 

We  were  somewhat  surprised  by  the  low  usage  of  Category  II  ILS  approaches.  About 
45  per  cent  of  the  pilots  reported  zero  Cat  II  approaches.  All  of  CAL’s  fleets  were 
qualified  for  Cat  II.  The  following  were  qualified  for  Cat  III  at  the  time  of  the  study  737- 
500,  757,  MD-80,  and  737-300.  Now  the  777  and  737NG  can  be  added  to  the  list,  and 
the  MD-80  is  Cat  III  qualified. 
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0  1  2  3  4  5  e* 

Number  of  VOR  Approaches 


FIGURE  IV-1 1 .  Frequency  of  VOR  approaches. 


FIGURE  IV-1 2.  Frequency  of  localizer  approaches. 
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FIGURE  IV-13.  Frequency  of  Category*!!  ILS 
approaches. 
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C.  PHASE  2  DATA 


In  this  section  we  shall  discuss  the  demographic  data  of  Phase  2.  This  phase  was 
timed  to  be  after  the  end  of  training  and  initial  operating  experience.  Volunteer  pilots 
were  sent  Q2  forms  approximately  3-4  months  after  they  entered  the  study.  This  period 
allowed  time  for  transition  training  (ground  school  and  simulator),  IOE,  vacation  time, 
and  at  worst  about  two  months  back  in  their  previous  seat  while  awaiting  a  757  seat 
(see  Figure  IV-14  and  15). 

As  the  study  moved  on  into  the  latter  half  of  the  1 990's,  this  became  less  of  a  factor. 
Pilots  went  straight  through  the  program  and  joined  the  line  without  interruption. 

Thus,  if  our  timing  was  right,  and  if  the  pilots  filled  out  the  Q2  form  promptly,  one  may 
think  of  the  second  phase  questionnaire  as  being  close  to  the  initial  point  of  a  pilot’s 
line  experience.  His  formal  training  was  complete,  and  he  would  now  be  learning 
through  on-the-job  training.  We  should  also  note  that  this  is  the  point,  early  in  our 
study,  at  which  our  sample  size  diminished,  due  to  the  bid-off  of  the  757,  due  largely  to 
what  were  perceived  as  poor  schedules. 

Post-Training  Assignment 

In  the  first  year  of  this  study  many  pilots  completed  training,  and  in  some  cases  IOE, 
and  then  had  to  be  assigned  to  their  former  aircraft  for  typically  two  months  until  a  757 
seat  was  available.  One  question  simply  asked  if  they  were  assigned  after  training  to  a 
757,  or  their  former  plane.  The  results  of  this  question  are  tabulated  below. 


757:  84  % 

Former  plane:  16% 

The  cockpit  they  returned  to  is  summarized  in  Figure  IV-14. 
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Aircraft  Assignment  After  Training 


FIGURE  IV-14.  Aircraft  assigned  to  following  757 
transition 


For  those  who  did  not  go  the  757  immediately,  the  number  of  months  of  reassignment 
to  their  former  plane,  before  moving  to  the  757,  is  shown  graphically  in  Figure  IV-15. 
This  was  in  part  due  to  the  difficulty  of  balancing  training  through-put  with  new  aircraft 
arrivals,  during  a  period  of  rapid  fleet  expansion.  No  fleet  manager  wants  to  get  caught 
short  of  flight  crews  with  new  aircraft  on  the  tarmac  The  obvious  solution  is  to  absorb 
some  costs  and  train  to  inventory,  hoping  to  make  use  of  the  excess  pilots  in  their  old 
plane  until  a  757  billet  is  available.  They  did  not  do  badly:  Figure  IV-14  and  the  data 
on  the  previous  page  show  that  the  vast  majority  of  pilots  went  directly  from  757 
training  to  the  757  line,  and  the  maximum  delay  for  those  who  did  not  was  three 
months.  The  following  ASRS  report  addresses  this  issue 


NARRATIVE.  Finished  checkout  F/O  6/89.  No  position  until  10/89.  Flew  simulator  in  9/89 
for  90-day  landing  currency.  You  could  say  the  find  points  of  working  the  FMC  had  escaped  my 
memory.  We  were  cruising  at  FL390  and  received  clearance  to  FL410.  Captain  loaded  in  MCP 
glare  shield  altitude~at  which  point  I  asked  how  he  input  the  data  for  the  climb,  neither 
monitoring  to  confirm  the  climb  to  FL410.  Several  mins  later  Center  asked  if  we  had  climbed. 
"No,  still  at  390  "  The  altitude  had  not  been  put  in  the  FMC,  and  we  were  navigating  with  VNAV 
and  LNAV.  Both  crew  members  low  experience  levels  in  type  contributory  to  the  altitude 
oversight.  Factors  affecting  performance:  1)  supervision  management  practice  of  putting  2 
inexperienced  crew  members  together,  or  2)  just  not  monitoring/keeping  track  of  crews'  levels  of 
experience;  and  3)  after  training  crew  member  on  advanced/automated  cockpit,  waiting  an 
extended  period  before  assignment  to  aircraft.  Fly  the  aircraft.  ACCESSION  NUMBER 
124912 
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FIGURE  IV-15.  Months  following  757  transition 
training  before  being  assigned  to  the  757.  Zero  months 
represents  those  going  directly  to  757  following 
training. 


Flying  Experience 

Pilots  were  asked  to  estimate  their  total  757  flying  time.  These  data  are  displayed  in 
Figure  IV-16.  It  was  expected  that  at  this  point  the  757  time  would  be  quite  low.  We 
estimated  that  the  pilot  would  have,  at  best,  200-300  hours.  Some  pilots  had  as  much 
as  500  hours.  The  very  high  times  shown  in  Figure  IV-16  can  only  be  from  pilots  who 
did  not  send  the  form  back  promptly,  and  amassed  flying  time  before  filling  out  at  least 
that  question.  This  graphic  will  be  displayed  again  in  this  chapter,  when  data  from  P3 
are  presented,  in  order  that  it  may  be  compared  to  the  flying  time  of  the  pilots  in  P3 
(over  a  year  later). 

On  the  following  page  we  have  two  bar  charts  showing  757  flying  time  at  Phase  2  and 
Phase  3.  Note  that  the  plots  are  on  different  scales.  On  the  top  graph  (Phase  2),  the 
bars  are  1 00  hours  apart;  on  the  lower  they  are  200  hours  apart. 


IV-24 


Total  Flying  Time  in  757 


Hours 


FIGURE  IV-16.  Total  flying  time  in  757,  at  Phase  2. 


FIGURE  IV-17.  Total  757  flying  time,  at  Phase  3 
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D.  PHASE  3  DATA 


The  third  and  final  phase  of  the  experiment  (P3)  was  designed  to  be  approximately  one 
year  after  IOE.  In  other  field  studies  that  we  have  conducted,  this  time  is  usually  found 
to  be  a  turning  point  at  which  the  pilot  starts  to  “feel  comfortable.”  Although  there  is 
always  more  to  be  learned,  at  this  time,  with  a  year’s  line  experience  behind  him,  the 
pilot  new  to  the  glass  cockpit  has  mastered  the  FMS  functionality,  autopilot  modes, 
display  modes,  etc.,  and  has  probably  also  mastered  the  “tricks”  of  line  flying  a  glass 
aircraft. 

Comfort  with  the  Aircraft 


In  this  study  the  pilots  appeared  to  “feel  comfortable"  (a  phrase  widely  used  by  pilots) 
much  earlier  than  our  previous  work  would  forecast.  Figure  IV-1 8  shows  the  result  of 
the  compound  question  in  Q3,  “Do  you  feel  ‘comfortable’  in  the  757  now?  (Y/N).  If  yes, 
how  long  after  you  went  on  the  line  did  it  take  (months)?”  As  to  the  first  question,  97  % 
reported  “yes”,  they  felt  comfortable.  The  durations  on  the  line  are  shown  in  Figure  IV- 
18.  Almost  half  of  the  respondents  reported  two  months  or  fewer,  and  a  very  small  per 
cent  offered  replies  of  over  six  months.  Such  favorable  results  are  probably  due  to  the 
user-friendliness  of  the  training  program.  The  high  confidence  and  high  regard  that  this 
cohort  had  for  the  transition  training  program  emerges  in  many  places  in  this  report: 
e.g.  attitude  probes,  interviews,  and  the  open-ended  questions  in  Q2  and  Q3. 


How  long  did  rt  take  you 
to  feel  comfortable  (months)7 


FIGURE  IV-1 8.  Months  (following  training)  before  pilot 
felt  “comfortable”  flying  the  757  in  line  operations. 
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Flying  Time,  Phase  2  and  Phase  3 


The  total  hours  of  757  flying  time  at  Phase  2  (actually  whenever  the  respondent  filled 
out  his  questionnaire)  is  depicted  in  Figure  IV-16.  Figure  IV-17,  the  757  time  at  Phase 
3  is  displayed  on  the  same  page  for  comparison.  Phase  2  was  designed  to  be 
approximately  3-4  months  after  transition  training.  Again,  the  calculation  of  the 
intervals  between  phases  is  at  the  mercy  of  the  pilot  volunteer  and  when  he  fills  out  this 
questionnaire.  For  PI  and  P2,  we  tailored  the  delivery  of  Q2  and  Q3  to  each  pilot, 
attempting  to  deliver  the  questionnaires  to  him  based  on  the  nominal  time  of  the 
phases.  Viewing  the  two  figures  together  allows  one  to  see  the  growth  in  757  flying 
time  during  the  (nominal)  year,  from  IOE  to  the  point  at  which  he  filled  out  the  Q3  form. 

In  the  following  narrative,  we  see  a  captain  who  has  only  been  on  the  aircraft  one 
month  who  handles  a  very  difficult  emergency,  and  attributes  his  success  to  the  training 
he  received,  as  well  as  the  competence  of  the  first  officer.  We  believe  the  reader  will 
agree  that  this  is  a  good  example  of  airmanship  by  a  captain  with  an  extremely  low 
time-in-type.  Note  that  it  is  obvious  that  this  report  is  from  an  Airbus,  as  the  term 
ECAM  is  used. 


NARRATIVE.  Aircraft  was  in  Cruise  at  FL290  due  to  the  prior  shutting  down  of  the 
number-two  pack  for  overhead  We  were  just  preparing  to  descend  to  cross  Bradford 
intersection  at  FL240  when  we  heard  a  possible  compressor  stall  and  the  aircraft  shook 
and  yawed.  We  got  confirmation  of  the  number-two  engine  failure  in  the  electronic 
controlled  aircraft  monitoring  ECAM  [an  Airbus  term  which  is  the  counterpart  of 
Boeing’s  EICAS]  ECAM  procedures  were  followed.  The  first-officer  was  flying  and  I 
allowed  him  to  continue  to  do  so.  I  did  not  elect  to  do  a  restart  as  the  EGT  was  climbing 
rapidly.  I  shut  down  the  engine  according  to  ECAM  and  used  the  fire  bottle  due  to  high 
and  rising  EGT.  Started  APU,  declared  emergency  with  ATC,  notified  Dispatch,  made 
public-announcement  to  passenger,  subsequently  lost  APU  before  landing,  wouldn't 
restart.  Elected  not  to  prepare  cabin  for  evacuation  and  weather  was  VFR.  Landed 
without  incident  at  Chicago  and  taxied  back  to  gate  as  all  hydraulic  systems  were 
operating  normally.  The  number-two  engine  compressor  section  had  failed  completely 
and  broken  up.  The  engine  was  replaced  by  maintenance.  I  was  fortunate  to  have  along  a 
very  competent  first-officer,  and  although  I  had  only  been  on  the  aircraft  a  month,  training 
had  prepared  me  very  well  to  handle  the  problem.  ACCESSION  NUMBER.  284470 
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V.  ANALYSIS  OF  QUESTIONNAIRE  DATA 


A.  METHODS  OF  ANALYSIS 


Because  most  of  the  variables  examined  in  this  study  were  categorical  (e.g.  captain 
versus  first  officer;  or  previous  glass  experience  versus  none),  the  data  collected  are 
best  suited  to  nonparametric  analysis.  Where  possible  these  methods  were  employed, 
and  interval  data  (such  as  age  and  flying  hours)  were  divided  into  categories  (see 
Chapter  IV).  In  this  chapter  we  shall  report  and  comment  on  those  data  that  were 
analyzed  and  subjected  to  statistical  tests.  Much  of  the  data  are  reported  in  Chapter  IV 
as  merely  descriptive  data,  not  suited  to  statistical  analysis  (e.g.  choice  of  favorite 
aircraft  in  company’s  fleet).  In  some  cases,  for  statistical  convenience,  the  data  are 
treated  as  being  on  an  interval  scale,  when  more  correctly  they  are  on  an  ordinal  scale. 
For  example,  the  intercorrelation  matrices  were  computed  using  the  Pearson  product 
moment  method,  which  properly  requires  interval  data,  but  is  widely  used  for  ordinal 
data,  such  as  responses  on  a  Likert  scale. 

Other  data  were  subjected  to  cross-tabulation  tests  (contingency  tables)  using  the  chi- 
square  distribution.  An  example  would  be  determining  if  there  is  a  relationship  between 
a  variable  which  we  have  called  “SEAT’  (captain  vs.  first  officer)  and  some  other 
categorical  variable  such  as  computer  usage  (yes/no).  Unfortunately  our  sample  size 
is  small  for  the  number  of  variables  examined,  and  some  compromises  with  statistical 
purity  were  made.  In  the  case  of  contingency  tables,  there  were  often  low  frequencies 
at  the  extreme  points  (corners  of  the  matrix),  so  the  results  may  be  inexact. 

In  the  case  of  the  intercorrelations  of  the  Likert  variables,  each  pair  taken  together 
potentially  produced  a  5-by-5  matrix.  In  some  cases  there  were  no  responses  at  all  for 
a  given  response  category,  thereby  reducing  the  matrix.  Usually  those  cases  involved 
one  or  both  of  the  two  extremes  where  the  respondent  could  “ strongly  agree”  or 
“ strongly  disagree”.  We  have  seen  in  this  study,  and  in  previous  ones,  a  tendency  on 
the  part  of  the  pilot  volunteers  to  avoid  the  extremes,  for  example,  the  graphs  on  page 
A-1 1 .  No  pilot  responded  in  the  "strongly  disagree”  category  in  response  to  that  probe. 
There  are  no  cases  in  our  data  where  there  were  fewer  than  four  non-zero  categories. 
Eleven  tests  involved  reduced  matrices  due  to  one  extreme  (either  “strongly  agree”  or 
“strongly  disagree”)  yielding  no  responses. 

Throughout  this  report  we  use  the  conventional  value  of  .05  for  the  statistical 
significance  level  (alpha,  or  probability  of  a  type-1  error).  In  interpreting  the  correlation 
matrices,  an  absolute  value  of  Pearson’s  r  >  .163  is  significant  for  n  =  147  for  two-tailed 
null  hypotheses.  For  n  =  146,  where  the  data  from  one  volunteer  may  be  missing,  the 
critical  value  is  very  slightly  higher  in  the  third  decimal  place. 
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B.  PHASE  1 


Intercorrelations 


For  each  of  the  three  questionnaires  (Q1,  Q2,  and  Q3),  an  intercorrelation  matrix  of  the 
responses  to  the  Likert  probes  was  computed  as  described  above.  Additionally  ten 
selected  demographic  variables  were  included  in  the  original  matrix,  but  are  not 
included  in  the  matrices  reported  here,  due  to  the  fact  that  their  inclusion  would  result 
in  a  vastly  expanded  matrix  A  copy  of  any  of  the  entire  intercorrelation  matrices 
mentioned  in  this  report  is  available  to  the  qualified  requester. 


The  size  of  the  entire  square,  symmetric  intercorrelation 
matrix  is  a  function  of  the  square  of  the  number  of  variables 
included;  a  single  echelon  of  the  symmetric  matrix  would 
contain,  for  N  variables,  N(N-1)/2  correlations.  If  the 
computer  program  prints  the  entire  square  matrix,  there  are 
N2  correlations. 


The  Q1  questionnaire  included  20  Likert  scale  probes,  resulting  in  190  correlations. 

Had  we  included  the  ten  demographic  variables,  there  would  be  465  correlations.  With 
our  statistical  software,  the  resulting  matrix  would  require  18  printed  pages. 

Accordingly  the  correlations  between  the  demographic  variables  and  the  Likerts  were 
examined  separately.  Some  of  the  terms  were  obvious  in  their  correlation  and  hence 
are  not  reported,  for  example  the  positive  correlations  between  total  flying  time  and 
flying  time  at  Continental,  or  some  of  the  obvious  correlations  of  variables  (e.g.  AGE,  or 
total  flying  time  --  TFTALL)  with  “SEAT”  (captain  vs.  first  officer).  We  chose  from  the 
list  of  1 1 ,  six  variables  to  be  included  in  the  correlation  matrix  for  Q1 .  There  were 
originally  ten  variables.  We  later  derived  GLASSNU  (see  next  page).  The  variables  are 
described  on  the  following  page. 
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AGE 

pilot’s  age  (months) 

SEAT 

position  on  757  (“1”  =  Capt.,  “2"  =  F/O) 

PCUSE 

does  pilot  use  PC  at  home  at  all  (“1"  =  yes,  “2"  =  no) 

TFTALL 

total  flying  time,  Continental  and  elsewhere  (hours) 

TFTCAL 

total  flying  time  at  Continental  (hours) 

GLASSNU 

had  the  pilot,  previous  to  757  transition,  flown  any  glass  cockpit? 
(“0"  =  no,  “1"  =  yes) 

The  following  list  are  demographic  variables  not  included  in  the  analysis.  See 
Chapter  IV  for  descriptive  statistics  of  these  variables. 


CHOICE 

Pilot’s  first  choice  of  aircraft  in  company’s  fleet 

LASTACFT 

Last  aircraft  flown  before  757  transition 

LASTMOS 

Months  on  last  aircraft  before  757  transition 

LASTS EAT 

Seat  occupied  before  757  transition  (Capt.  vs.  F/O) 

TABLE  V-1.  Demographic  variables  on  Questionnaire  1. 

We  shall  now  examine  some  of  the  variables  and  their  relationships. 

Pilots’  Age 

There  has  been  considerable  interest  in  both  the  airline  community  and  the  research 
community  on  the  effect  of  chronological  age  of  the  pilot  going  through  his  first  glass 
transition.  Much  of  what  was  said  was  based  on  unsupported,  usually  negative 
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stereotypes  of  older  workers  -  that  the  senior  captains  were  not  abreast  of  technology, 
since  they  did  not  grow  up  in  the  computer  age,  and  that  they  were  mentally 
unadaptable  to  the  high  tech  cockpit.  We  do  not  know  of  any  research  on  the  topic  of 
age  and  transition  to  glass.  There  has  been  considerable  interest  in  pilot  age  in  the  last 
three  decades  due  to  the  legal  requirement  that  pilots  flying  under  FAR  121  retire  at 
age  60  (the  “age  60  rule”).  But  the  research  to  support  that  rule  pre-dated  the  era  of 
the  FMS  cockpits. 


FIGURE  V-1.  Example  of  generally  accepted  probe 
which  correlated  with  age  of  pilot. 


Training  personnel  spoke  of  apprehension  borne  by  older  pilots.  In  our  interviews  at 
other  airlines  with  pilots  in  glass  transition  for  the  first  time,  there  was  frequent 
expression  of  apprehension  concerning  the  demands  of  the  transition  training, 
particularly  with  respect  to  their  lack  of  computer  skills.  These  concerns  always  seem 
to  come  from  captains:  investigators  did  not  encounter,  in  previous  studies, 
apprehension  on  the  part  of  the  first  officers  making  the  transition.  In  our  interviews 
with  pilots  and  instructors  in  the  current  study,  we  have  heard  less  of  this.  The  graph 
(8A)  from  Questionnaire  1  may  be  instructive.  It  is  shown  as  Figure  11-1.  Only  about 
10%  of  the  respondents  expressed  agreement  with  the  probe.  We  see  in  this  graph  a 
rather  strong  rejection  of  the  opinion  that  pilots  arrive  at  their  transition  training  filled 
with  apprehension.  This  subject  is  also  discussed  in  Chapter  II. 

Chi-square  tests  of  the  20  contingency  tables  of  attitude  response  vs.  seat  (captain  and 
F/O)  all  resulted  in  negative  findings.  We  have  found  no  difference  between  captains 
and  first  officers  in  replies  to  this  or  any  other  attitude  probes,  in  any  of  the  three 
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questionnaires,  or  on  questions  about  computer  usage.  In  brief,  we  have  found  no 
differences  between  captains  and  first  officers  in  the  757  program,  except  obvious 
factors  such  as  age  and  flying  experience.  Therefore  in  further  analyses  we  have 
aggregated  the  data  of  the  two  positions  into  one. 

Our  inquiries  in  previous  studies  (Wiener,  1989)  into  the  influence  of  age  revealed  that 
if  there  is  any  reliable  generalization,  it  is  that  the  older  pilots  seemed  to  get  off  to  a 
slow  start  in  early  days  of  ground  school,  having  a  slight  amount  of  trouble  mastering 
some  of  the  new  concepts,  compared  to  the  younger  first  officers,  who  were  presumed 
to  be  “techies”,  skilled  in  digital  concepts  and  operations.  After  this  initial  period,  the 
captains,  drawing  on  their  vast  experience  and  airmanship,  caught  up  and  by  the  time 
they  reached  the  simulator,  were  performing  at  a  high  level.  It  was  unfortunate  that  the 
training  book  data  did  not  work  out.  They  might  have  provided  somewhat  objective 
information  on  the  effect  of  age  during  training. 

AGE  correlated  significantly  with  one  Likert  probe,  No.  A12,  “I  have  no  trouble  staying 
‘ahead  of  the  plane’”.  (See  Figure  V-1 ,  previous  page.)  Since  the  Likert  scale,  when 
treated  as  an  interval  scale,  goes  from  1  to  5  as  it  goes  from  strongly  agree  to  strongly 
disagree,  a  negative  correlation  means  that  high  age  goes  with  low  Likert  values 
(approval  of  the  probe).  In  this  case,  the  older  the  pilot,  the  more  approving  he  is  of  the 
statement  that  he  can  easily  stay  ahead  of  the  plane.  The  younger  pilots  may  have 
some  reservations  about  their  own  abilities. 

For  obvious  reasons  the  variable  SEAT  (captain  vs.  F/O)  is  highly  (negatively) 
correlated  with  AGE  (r  =  -.54).  In  SEAT,  as  we  have  indicated,  captains  are  coded  as 
“T's  and  first  officers  as  “2"s.  The  lower  index  number  (captains)  is  associated  with 
higher  age.  For  this  reason,  SEAT  also  is  positively  correlated  with  A12. 

In  summary,  we  have  not  produced  any  evidence  on  differences  due  to  the  trainees’ 
age.  One  significant  Likert,  and  somewhat  obvious  correlations,  is  the  best  we  can 
offer.  The  age  question  will  have  to  await  perhaps  a  simulator  study  in  which 
dependent  variables  can  be  carefully  measured  and  examined  with  respect  to  the 
pilots’  ages. 

Seat 


The  variable  SEAT  refers  to  the  seat  that  was  bid  for  757  training,  “1"  for  captain.  In 
most  cases  the  seat  in  the  757  bid  was  the  same  as  that  held  at  the  time  of  bid.  A  few 
senior  first  officers  bid  for  757  captain  seats,  and  made  the  transition  and  upgrade  at 
the  same  time.  SEAT  is  highly  (negatively)  correlated  with  AGE  (r  =  -.54)  due  to 
seniority  considerations.  SEAT  is  also  correlated  with  A12  (r  =  .17),  possibly  through 
its  correlation  with  AGE.  It  also  correlates  (r  =  .18)  with  A10  (“I  am  not  concerned 
about  making  errors,  as  long  as  we  follow  procedures  and  checklists").  This  indicates 
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that  the  captains  are  more  accepting  of  the  probe  than  the  first  officers.  This  finding, 
and  the  one  indicating  a  positive  correlation  between  age  and  A12,  suggest  a  degree  of 
caution  and  conservatism  on  the  part  of  the  first  officers,  and  self-confidence  on  the 
part  of  the  captains.  This  runs  counter  to  the  popularly  held  stereotype  of  the  ultra- 
conservative  captain. 

The  relationship  between  seat  and  attitude  was  also  tested  by  forming  a  2-by-5 
contingency  table  (2-by-4  in  those  cases  where  an  extreme  [SA  or  SD]  had  zero 
entries),  with  attitude  choice  as  a  column  variable  and  seat  as  a  row  variable.  The  chi- 
square  contingency  coefficient  was  computed  and  tested  for  all  20  probes  None 
resulted  in  a  rejection  of  the  null  hypothesis  of  row/column  independence. 

Flying  Experience 

The  two  measures  of  flying  time,  TFTALL  and  TFTCAL  naturally  correlate  highly  with 
each  other  (r  =  .81).  This  correlation  is  obvious,  since  the  pilot’s  total  flying  time, 
TFTALL,  contains  the  value  of  the  variable  TFTCAL,  his  flying  time  at  Continental. 

They  also  correlate,  as  one  would  expect,  with  SEAT  and  AGE. 

TFTALL  correlates  (r  =  .16)  with  A18  (“Continental’s  CRM  training  has  been  helpful  to 
me”),  positive  correlation  indicating  that  pilots  with  high  flying  hours  tend  to  take  a  less 
favorable  view  of  the  CRM  training.  This  is  probably  due  to  the  correlation  with  rank:  it 
would  indicate  that  low-time  pilots  (mostly  first  officers)  are  more  accepting  of  CRM 
than  captains,  which  is  the  experience  at  most  airlines.  It  is  interesting,  and  not  easily 
explained,  that  the  correlation  of  TFTCAL  with  this  probe  was  very  small.  Total  flying 
time  appears  not  to  be  a  particularly  fruitful  variable.  In  the  discussion  to  follow,  of 
Questionnaire  2  and  3,  the  interest  will  shift  to  757  flying  time  as  a  predictor  variable. 

GLASSNU  was  a  derived  variable,  based  on  the  question  about  prior  glass  experience. 
We  created  this  variable  in  order  to  test  hypotheses  about  attitudes  as  a  function  of 
having  flown  or  not  flown  glass  aircraft  before  757  transition.  The  variable  we  created 
was  a  “(0,  1)"  variable:  it  recorded  only  yes  (1)  or  no  (0),  did  the  pilot  have  prior  glass 
experience?,  and  does  not  reflect  the  amount  of  glass  flying  time. 

The  influence  of  past  glass  experience  was  tested  by  forming  a  contingency  table  for 
each  probe  (as  in  the  SEAT  variable  above),  and  performing  a  chi-square  test  on  each. 
This  resulted  in  three  rejections  of  the  null  hypothesis  of  row/column  independence. 

The  contingency  tables  for  the  three  are  displayed  below. 
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A7. 


In  the  aircraft  that  I  am  presently  flying,  it  is  easy  for  the  captain 
to  monitor  and  supervise  the  first  officer. 
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GLASS 
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Chi-square  =  8.12,  df  =  3,  p  <  .05 


A16.  I  am  concerned  about  the  reliability  of  some  of  the  automation  equipment. 
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Chi-square  =  21 .05,  df  =  4,  p  <  .001 


A17.  I  am  concerned  about  the  lack  of  time  to  look  outside  the  cockpit  for 
other  aircraft. 
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Chi-square  =  10.47,  df  =  4,  p  <  .05 
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The  interpretation  of  the  entries  in  the  matrix  is  up  to  the  reader.  It  would  appear  that  in 
probe  A7  the  glass-experienced  pilots  had  a  narrow  range  of  opinion,  mostly  agreeing 
with  the  sense  of  the  probe,  and  a  small  number  neutral.  Those  without  glass 
experience  showed  more  variability,  though  the  distributions  were  centered  at  about  the 
same  place.  There  was  very  high  agreement  by  the  non-glass  group  -  only  four  out  of 
109  disagreed  with  the  probe. 

In  A16  the  non-glass  pilots  showed  a  fairly  symmetrical  distribution,  while  the  glass- 
experienced  pilots  had  rather  strong  disagreement  with  the  probe,  with  29  on  the 
disagree  side  and  5  on  the  agree  side.  Perhaps  their  experience  with  glass  cockpit 
had  relieved  some  of  the  apprehension  of  those  making  their  first  transition  to  glass. 

In  A17  the  glass  group  was  symmetrically  divided  over  the  range,  with  most  responding 
agree  or  disagree,  and  few  extreme  or  neutral.  The  non-glass  showed  somewhat  the 
same  pattern,  but  more  neutral  choices.  It  would  appear  that  the  non-glass  pilots  were 
somewhat  more  concerned  about  heads  up  time  than  the  glass  pilots. 

Home  Computer  Usage 

Since  the  introduction  of  the  FMS  into  airline  fleets  there  has  been  a  persistent  belief 
that  pilots  who  own  a  home  computer  profit  from  this  experience.  It  was  further 
assumed  that  it  is  first  officers  who  have  this  exposure,  giving  the  “computer  literate” 
first  officer,  if  not  an  advantage,  at  least  some  compensation  for  the  captain’s  greater 
aviation  experience.  We  again  state  that  this  entire  line  of  reasoning  has  been  based 
on  assumptions  and  beliefs,  not  on  empirical  data. 

In  an  earlier  NASA  report  (Wiener,  et  al.,  1991,  p  25),  the  question  of  ownership  of 
home  computers  was  raised.  Of  the  captains,  7 1  %  responded  yes,  and  for  the  first 
officers  it  was  50%.  For  this  sample  size  (n  =  73),  the  difference  was  not  statistically 
significant. 

In  this  study  we  corrected  the  mistake  we  had  made  in  earlier  studies  by  asking  not 
about  ownership,  but  usage  “Do  you  use  a  personal  computer  at  home?”  (PCUSE). 
The  second  question  (PCFREQ)  dealt  with  how  often  it  was  used.  The  statistical  test 
involved  a  2-by-4  contingency  table  (see  Table  IV-la).  Once  again  we  found  no 
difference  between  the  responses  of  the  captains  and  first  officers.  Thus  we  are 
convinced  that  the  myth  of  the  computer-  literate  first  officer  and  the  computer-naive 
captain  is  unsupported.  Whatever  problems  captains  may  have  in  transition  to  glass, 
compared  to  the  first  officers,  is  probably  not  due  to  differences  in  home  computer 
experience. 

There  were  two  significant  correlations  to  report  PCFREQ  correlated  significantly  with 
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A1  (“Flying  today  is  more  challenging  than  ever”)  (r  =  -.18).  Since  the  correlation 
coefficient  is  negative,  it  indicates  that  frequent  PC  users  (low  index  numbers)  tended 
to  have  low  approval  (high  Likert  scale  response  values)  of  this  probe.  We  find  it 
difficult  to  interpret  this  result. 

Likewise,  PCUSE  correlated  significantly  with  A16  (“I  am  concerned  about  the  reliability 
of  some  of  the  automation  equipment .”)  (r  =  -.19).  Here  the  result  may  be  more  clear. 
The  correlation  coefficient  is  negative,  indicating  that  PC  users  (“1")  tended  to  give 
higher  Likert  responses  (disapproval  of  the  probe).  PC  users  may  indeed  be  more 
accepting  of  automation  technology,  even  its  faults,  than  non-users  (“2"). 

Summary 

These  data,  and  the  descriptive  data  presented  in  Chapter  IV  have  not  produced  any 
startling  results,  but  together  paint  a  mosaic  of  the  pilots'  attitudes  toward  transition 
training  in  a  new  technology  aircraft.  Further  details  will  be  found  in  crew  members' 
responses  to  the  open-ended  questions,  presented  in  Chapter  VI.  This  completes  the 
discussion  of  Phase  1  by  itself.  We  now  turn  to  Phase  2,  and  to  comparisons  between 
Phase  1  and  Phase  2. 
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C.  PHASE  2 


The  second  phase  of  the  experimental  design  was  timed  to  be  about  four  months  after 
ground  school,  following  all  training  including  IOE,  and  assignment  to  a  base  to  fly  the 
757  line. 

Much  of  the  data  are  summarized  graphically  in  Chapter  IV.  The  second  phase 
questionnaire,  Q2,  included  a  small  number  of  demographic  variables,  four  open-ended 
questions  (which  are  analyzed  in  Chapter  VI),  and  a  24-item  attitude  scale.  The  24 
items  included  the  20  utilized  in  PI ,  plus  four  new  items  (21-24).  The  following 
demographic  variables  were  included. 


TIME757 

Total  hours  757  time 

CBASE 

Current  base 

ACASSIGN 

The  plane  the  pilot  that  was  assigned  to  after  training 
(757  or  otherwise) 

MORTRN 

The  number  of  months  assigned  to  plane  other  than  757, 
following  transition  training,  prior  to  assignment  to  757  line. 

PROB757 

This  variable  sought  to  measure  problems  encountered  by 
pilots  returning  to  their  old  aircraft  to  await  a  757  assignment. 
Since  so  few  fell  into  this  category  (see  Figure  IV-14),  we  have 
not  used  this  variable. 

Intercorrelations 


The  intercorrelation  matrix  of  the  24  Likert  scale  variables,  as  well  as  the  demographic 
variable  TIME757  was  computed.  The  intercorrelation  matrix  is  too  large  to  include  in 
this  report;  it  is  available  on  request. 

ACASSIGN  and  MORTRN 

These  variables  are  discussed  and  graphics  depicting  the  variables  are  displayed  in 
this  chapter.  The  graphics  of  ACASSIGN  (Figure  IV-14)  and  MORTRN  (Figure  IV-15) 
indicate  that  only  about  12  per  cent  of  the  sample  was  unable  to  move  directly  into  757 
line  assignments.  In  the  first  few  classes  the  figure  was  somewhat  greater,  as 
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deliveries  did  not  keep  up  with  pilot  training.  Later  the  opposite  was  true:  deliveries  ran 
ahead  of  pilot  training,  training  was  accelerated,  and  new  757  pilots  went  to  the  line 
without  delay. 

TIME757 

The  variable  TIME757,  the  number  of  hours  of  757  time  accumulated  up  to  the 
completion  of  the  P2  questionnaire,  is  displayed  graphically  in  Chapter  IV  as  Figure  IV- 
16.  It  is  displayed  again  in  the  next  sub-chapter  of  this  chapter  along  with  the  same 
question  for  P3,  so  that  the  growth  in  flying  time  accumulated  in  P2  and  P3,  over  a  12- 
14  month  period  can  be  compared  (Figures  IV-16  and  IV-17). 

It  would  be  interesting  if  this  variable  correlated  with  various  attitude  scale  scores.  The 
correlation  between  TIME757  and  each  of  the  24  Likert  scale  variables  was  computed, 
and  only  one  was  significant:  the  correlation  with  A18  (“Continental’s  CRM  program  has 
been  helpful  to  me”)  was  0.22  (p  <  .025).  (Under  the  null  hypothesis  of  zero  correlation 
between  two  variables,  for  n  =  102,  an  absolute  value  of  Pearson’s  r  greater  than  .196 


FIGURE  V-2.  Attitude  toward  company’s  CRM  program. 


significant  at  the  .05  level,  two-tailed  test).  Since  the  correlation  is  positive,  this 
indicates  that  pilots  with  higher  flying  time  in  the  757  tend  somewhat  more  to  reject  the 
probe.  This  is  consistent  with  the  finding  from  PI  that  there  was  likewise  a  significant 
correlation  between  total  flying  time  (TFTALL)  and  the  A18  (CRM)  probe  (see  page  V- 
6).  The  history,  background,  and  theoretical  foundations  of  Continental’s  CRM 
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program  is  discussed  in  Chapter  VII. 

PROB757 

On  Q2  there  were  questions  about  what  plane  the  pilot  returned  to  after  757  transition, 
if  he  could  not  be  assigned  to  the  757.  As  we  indicated  previously,  only  about  16%  of 
the  sample  returned  to  their  previous  plane  rather  than  the  757,  and  this  sub-sample 
was  too  small  to  be  worthy  of  statistical  testing.  For  confirmation,  see  Figure  IV-14. 


GLASSNU 

24  contingency  tables  were  formed,  using  the  derived  index  GLASSNU  and  each 
attitude  probe.  These  resulted  in  mostly  2-by-5  tables,  in  a  few  cases  2-by-4.  Each 
was  tested  using  the  chi-square  test.  None  was  significant.  We  can  conclude  that 
pilots  who  had  formerly  flown  glass  cockpits  did  not  differ  in  attitude  toward  training  and 
automation  from  those  who  had  not. 


SEAT 

Contingency  tables  2-by-5  (or  2-by-4)  were  formed  to  test  the  variable  SEAT  (captain 
vs.  first  officer)  against  the  24  Likert  attitude  probes.  None  was  significant.  We  again 
see  that  the  attitude  of  captains  and  first  officers  did  not  differ  in  this  sample. 

Summary 

The  attitude  and  demographic  data  from  Phase  2  have  been  analyzed  in  the  foregoing 
sub-chapter.  We  now  turn  to  comparisons  of  the  attitude  data  between  PI  and  P2. 
This  is  the  longitudinal  analysis.  Out  of  this  analysis  will  come  a  comparison  of 
responses  in  the  two  phases,  which  will  tell  us  whether  attitudes  shifted  between 
examination  during  Phase  1  (sign-up)  and  second  Phase  2  (post-lOE). 


D.  COMPARISON  OF  PHASE  1  AND  PHASE  2  ATTITUDE  RESULTS 


In  this  sub-chapter  we  examine  the  attitude  results  from  PI  and  P2,  to  determine 
whether  there  has  been  an  attitude  shift  during  the  3-4  month  period  between  the  times 
when  the  pilots  filled  out  Q1  and  Q2.  We  are  particularly  looking  for  shifts  in  attitudes 
toward  training  and  toward  automation  in  general.  A  shift  would  be  indicated  by  finding 
differences  in  a  pilot’s  responses  to  the  same  question  asked  during  the  two  periods, 
that  is,  an  inconsistency  between  response  on  P2  and  P3. 
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PI  vs.  P2  Comparisons:  Corresponding  Questions 


The  following  adjustment  was  made  in  numbering  of  Q1  and  Q2  probes.  Q1  No.  8  (“/ 
am  very  apprehensive  about  going  through  this  transition *)  was  inappropriate  for  Q2 
and  Q3,  so  no  comparison  with  it  was  possible.  Q1  No.  14  was  moved  to  take  its 
place.  On  page  A-8  the  probes  14a,  8b,  and  8c  are  shown  graphically.  Although  the 
numbers  are  different,  the  probes  are  the  same.  Otherwise  each  of  the  first  20  pages 
of  Appendix  A  show  the  three  graphs  representing  the  three  phases  in  proper  order 
(Probes  1-13,  and  15-20).  Pages  A-21  through  A-24  display,  two  to  a  page,  the  results 
of  the  four  probes  used  on  Q2  and  Q3,  but  not  Q1 . 

The  Test  Statistic:  Marginal  Homogeneity 

Since  the  test  statistic  may  not  be  familiar  to  all  of  the  readers,  we  shall  describe  it 
briefly.  The  statistical  measure  is  called  the  marginal  homogeneity  test.  It  is  an 
extension  and  generalization  of  the  familiar  McNemar  repeated  measures  test  with  two 
response  categories  (2-by-2).  The  McNemar  problem  is  generalized  to  K-by-K  matrix 
for  K  response  categories.  There  is  also  a  K-by-K  categories  test  attributed  to  Bowker, 
used  in  a  previous  field  study  (Wiener,  1989).  For  the  mathematical  development  of 
the  marginal  homogeneity  test,  see  Agresti  (1990). 

The  data  must  be  categorical  and  ordered  Arbitrarily  the  first  phase  (PI )  responses 
are  assigned  to  rows,  the  second  phase  (P2)  to  columns.  Thus  for  the  attitude  data,  a 
5-by-5  matrix  (or  in  some  cases  smaller)  is  produced,  with  cell  ij  representing  a 
response  of  /  to  the  first  application  of  the  probe  (PI ),  and  j  to  the  second  (P2).  If  the 
pilot  responds  the  same  on  both  applications  of  the  probe,  the  tally  will  go  in  the  main 
diagonal  (I  =  J).  If  there  is  a  shift  in  opinion,  more  cases  will  fall  off  the  main  diagonal. 

Page  C-1  is  repeated  on  the  following  page.  The  probe  is  No.  1 ,  “Flying  today  is  more 
challenging  than  ever.”  Looking  at  the  main  diagonal,  15  pilots  chose  the  “strongly 
agree”  response  category  on  both  Q1  and  Q2,  21  chose  “agree”,  etc.  The  off-diagonal 
tallies  indicate  shifts  in  attitude  between  the  first  to  the  second  polling.  Using  the  same 
example,  nine  pilots  changed  their  response  from  “strongly  agree"  to  “agree”.  If  there 
were  no  changes,  the  entire  tally  would  be  contained  in  the  main  diagonal.  The 
greater  the  change  in  attitude,  the  further  the  tally  would  fall  from  the  main  diagonal.  In 
the  example,  four  pilots  changed  their  attitude  response  from  “strongly  agree”  to 
“disagree.”  These  were  large  defections  from  the  initial  (P2)  position,  but  no  full-scale 
changes  (from  “strongly  agree”  to  “strongly  agree”,  or  vice-versa. 
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Phase 


Phase  2 
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Results:  PI  vs.  P2 


Under  the  two-tail  null  hypothesis  of  no  change  in  attitude,  the  responses  should  be 
clustered  on  or  near  the  main  diagonal.  A  large  number  of  off-diagonal  entries  (in 
either  direction  under  a  two-tail  null  hypothesis)  would  lead  to  a  rejection.  For  a  one-tail 
hypothesis,  the  direction  of  deviation  from  the  first  phase  to  the  second  is  specified.  We 
used  the  test  on  the  20  probes  in  common  to  PI  and  P2,  and  the  24  probes  in  P2  and 
P3.  In  Appendix  C  we  have  provided  the  matrix  for  only  those  probes  that  were 
statistically  significant.  Along  with  the  response  matrix  we  have  provided  a  graphic 
displaying  the  frequency  of  response  for  each  of  the  five  response  categories.  These 
two  figures,  although  they  use  the  same  data,  do  not  display  the  same  information. 

The  bar  graph  shows  trends  of  groups,  not  the  choices  of  individual  pilots. 

Table  V-3  lists  the  eight  significant  marginal  homogeneity  tests  from  Phase  1  compared 
to  Phase  2,  and  indicates  the  nature  of  the  change.  The  full  text  of  the  probes  can  be 
found  in  Appendix  D.  The  movement  of  response  from  Phase  1  to  Phase  2  in  the 
attitude  questionnaires  can  be  seen  graphically  in  Appendix  C. 

There  is  no  consistent  pattern  in  the  movement  toward  disagreement  with  the  probes, 
charted  from  PI  to  P2.  Some  of  the  probes  are  positively  stated  toward  automation 
(e.g.  No.  3),  and  some  are  negative  (e.g.  No.  2).  As  the  pilots  repeat  the  questionnaire 
in  P2,  approximately  three  to  four  months  after  the  first  set  of  responses  in  PI ,  a 
movement  toward  less  agreement  with  the  probes  does  not  portray  a  consistent  attitude 
toward  automation.  The  pilot  changes  his  choice  toward  less  agreement  with  No.  1 1 , 
looking  forward  to  more  automation,  and  likewise  changes  toward  less  agreement  with 
the  more  negative  No.  13. 

Perhaps  we  will  find  the  next  set  of  marginal  homogeneity  tests,  for  P2  vs.  P3,  more 
instructive.  In  those  tests  the  pilots  will  have  had  some  flying  experience  in  the  757, 
limited  to  a  few  months  in  P2,  and  over  a  year’s  worth  in  P3.  In  any  event,  it  is 
interesting  to  note  the  volatility  of  opinion  in  PI  vs.  P2,  that  eight  out  of  20  probes 
should  result  in  statistically  significant  changes  of  opinion,  even  if  we  cannot  find  a 
consistency  or  an  underlying  meaning  in  these  opinion  shifts.  When  we  compare 
opinions  in  P  2  and  P  3,  we  would  expect  more  stability  of  opinion,  that  is,  fewer 
significant  changes. 
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TABLE  V-3.  Statistically  significant  marginal  homogeneity  tests. 


Probe 

Abbreviated  Text 

Movement:  PI  to  P2 

1 

Flying  is  more  challenging  than  ever 

Toward  disagreement 

2 

Concerned  that  automation  will  cause 
skill  loss 

Toward  disagreement 

3 

Automation  leads  to  safer  operations 

Toward  disagreement 

7 

Easy  for  captain  to  monitor  first  officer 

Toward  disagreement 

11 

Look  forward  to  more  automation 

Toward  disagreement 

13 

Too  much  workload  terminal  area 

Toward  disagreement 

15 

Easy  to  bust  altitude  today’s  environment 

Toward  disagreement 

17 

Concerned  about  time  to  look  outside 

Toward  disagreement 
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E.  PHASE  3 


Phase  3,  the  final  phase  of  the  longitudinal  study,  was  designed  to  collect  data  from  the 
remaining  volunteers  at  a  time  when  they  had  about  one  year  of  line  experience,  or 
about  16-18  months  after  initially  joining  the  study  on  the  first  day  of  ground  school  for 
757  transition.  We  felt,  based  on  past  experience  in  field  studies,  that  at  this  time 
opinions  would  have  solidified,  and  would  probably  not  change  appreciably  if  the 
interval  between  P2  and  P3  were  extended.  Also,  we  would  expect  that  at  this  time  the 
pilots  would  feel  “comfortable”,  a  word  widely  used  by  pilots  to  describe  their  feelings  at 
some  experience  level  after  transitioning  to  a  new  aircraft.  To  be  “comfortable"  in  the 
new  aircraft  would  mean  that  the  pilot  was  free  of  apprehensions  about  his  ability  to 
manage  the  cockpit  and  particularly  the  automated  features,  to  be  able  to  respond 
appropriately  to  non-normal  situations,  and  in  brief  to  feel  at  home,  relaxed,  self- 
confident,  and  in  command  of  his  own  abilities.. 

Phase  3  was  designed  mainly  to  obtain  the  final  data  on  the  24  attitude  probes.  There 
is  a  minimal  amount  of  demographic  data  sought  on  the  questionnaire  (see  Appendix 
D),  and  a  minimal  number  of  hypotheses  to  be  tested.  Most  of  the  data  are  merely 
descriptive 

Miscellaneous  Questions 

As  to  the  "comfort”  dimension,  91  of  the  94  valid  answers  were  “yes”  to  the  question 
“Do  you  feel  comfortable’  in  the  757  now?” 

The  data  for  the  pilots’  current  base  are  tabulated  below.  The  number  of  responses  to 
this  question  of  IAH  and  EWR  are  about  proportional  to  the  757  flying  time  assigned  to 
those  two  bases.  The  desirability,  from  the  pilots’  view,  of  Continental’s  various  bases 
is  discussed  elsewhere.  The  Guam  757  base  was  essentially  ignored  in  this  study, 
since  it  was  formed  after  the  study  began. 


IAH  (Houston) 
EWR  (Newark) 
LAX  (Los  Angeles) 
Other 


33 

56 

1 

6 


Total 


96 
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Cockpit  Positions  Held 


The  cockpit  positions  held  by  the  volunteer  pilots  is  shown  in  Table  V-4.  Note  that  the 
96  pilots  represented  by  this  table  were  the  larger  group  (82  per  cent)  who  were  still 
flying  the  757;  the  remainder  were  flying  other  aircraft.  As  a  percentage  of  the  149 
original  volunteers,  this  group  is  probably  under-represented  due  to  the  fact  that  many 
pilots,  after  bidding  off  the  757,  dropped  out  of  the  study  and  did  not  send  in  Q3. 


Table  V-4.  Current  Aircraft  and  Seat  of  96  Pilots 


CURRENT 

AIRCRAFT 

CART. 

m 

.  ‘ .  ; 

TOTAL 

DC-9 

i 

0 

1 

MD-80 

i 

0 

1 

737-100 

i 

0 

1 

737-300 

2 

1 

3 

727 

2 

2 

4 

DC-10 

4 

2 

6 

Other 

1 

0 

1 

757 

46 

33 

79 

Total 

58  :  1  38  j 

\ 
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Contingency  Tests 


To  test  the  null  hypothesis  that  attitudes,  as  measured  by  our  Likert  scales,  are  not 
related  to  SEAT  (position  in  the  cockpit),  2-by-5  (in  some  cases  2-by-4)  contingency 
tables  were  cast,  as  in  Phase  2.  Chi-square  tests  were  performed  on  the  24  resulting 
tables  (one  for  each  attitude  probe).  None  was  significant.  We  must  again  conclude 
that  captains  and  first  officers  in  this  sample  saw  things  alike. 

Similar  tables  were  set  up  using  the  derived  variable  GLASSNU  (previous  glass 
experience,  or  none)  as  one  variable,  responses  on  the  Likert  scales  the  other.  None 
was  significant.  We  conclude  that  pilots  with  past  experience  on  glass  aircraft,  and 
those  with  no  such  experience,  held  similar  attitudes  as  measured  by  our  scales. 

Intercorrelations 


An  intercorrelation  matrix  containing  responses  on  the  24  attitude  scales  was  formed, 
but  was  not  examined  statistically. 


F.  COMPARISON  OF  PHASE  2  AND  PHASE  3  ATTITUDE  RESULTS 


As  in  sub-chapter  D,  we  shall  now  examine  the  24  attitude  scales  for  possible  shifts  in 
attitude  from  P2  to  P3,  using  the  marginal  homogeneity  tests. 

Of  the  24  attitude  scale  items,  three  showed  significant  changes  from  P2  to  P3,  as 
summarized  in  Table  V-5  below. 


TABLE  V-5.  Statistically  significant  marginal  homogeneity  tests,  P2  vs.  P3. 


Probe 

Z 

Text 

Movement  P2  to  P3 

17 

-2.3 

1  am  concerned  about  the  lack  of  time  to 
look  outside  the  cockpit  for  other  aircraft 

Toward  agreement 

18 

-2.7 

Continental’s  CRM  program  has  been  helpful 

Toward  agreement 

24 

3.0 

There  are  still  modes  and  features  of  the 

757  that  1  do  not  understand 

Toward 

disagreement 
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As  Table  V-5  and  the  figures  in  Appendix  E  indicate,  there  was  a  significant  shift  of 
opinion  in  the  pilots  in  the  roughly  14  months  between  Phases  2  and  3.  Numbers  17 
and  24  indicate  movement  favorable  to  the  757  flight  guidance  system,  and  perhaps 
toward  automated  flight  in  general.  In  previous  field  studies  these  are  common  worries 
of  the  new  FMS  aircraft  pilots:  sufficient  time  for  extra-cockpit  scanning;  and  mode 
confusion,  as  it  has  come  to  be  called. 

We  observed  also  an  increasingly  favorable  view  toward  the  company’s  CRM  program. 
We  can  only  speculate  as  to  the  reason  for  this.  It  is  most  likely  due  to  the  emphasis 
put  on  CRM  throughout  the  757  transition  training.  Both  fleet  managers  insisted  that 
CRM  be  taught  and  included  as  part  and  parcel  of  the  flight  training,  not  as  a  separate 
block  of  instruction  (see  quotation,  top  of  page  VII-1 ).  Some  authors  have  stated 
previously  (Wiener,  1989)  that  good  CRM  practices  are  even  more  essential  in  the 
automated  than  in  the  traditional  cockpits.  This  point  was  emphasized  from  the  first 
day  (H-CAT  training).  Much  of  the  credit  for  the  emphasis  on  blending  CRM  with  flight 
training  goes  to  instructors  at  Boeing.  It  was  there  that  the  early  cadres  of  Continental 
instructors  first  encountered  this  method  of  training.  Typically  flight  training  and  CRM 
are  taught  as  two  worlds  apart.  Seeing  the  merit  of  the  Boeing  approach,  it  was 
transported  back  to  Houston  and  made  part  of  the  flight  training  program.  Presumably, 
over  the  period  between  filling  out  Q1  and  Q2,  a  space  of  over  a  year,  the  pilots  had  an 
opportunity  to  practice  their  CRM  skills  in  a  demanding,  automated  environment.  This 
perhaps  accounts  for  the  shift  toward  a  more  positive  response  to  probe  number  1 8. 

Usage  of  HSI  Modes 

At  the  request  of  Continental  flight  management,  we  included  a  question  in  the  third 
(final)  questionnaire  (Q3),  asking  the  757  pilots  the  percentage  of  time  they  used,  in 
flight,  the  four  HSI  modes  (map,  expanded  VOR,  expanded  ILS,  and  compass  rose). 
See  Page  D-10. 

The  map  mode  was  used  about  95%  of  the  time,  and  the  remaining  5%  was  about 
evenly  split  between  the  other  three  modes.  As  we  have  noted  in  previous  studies  (e.g. 
Wiener,  1989),  most  pilots  will,  if  unimpeded,  fly  an  entire  trip  with  the  HSI  in  map 
mode.  Why  not?  It  is  a  spectacular  example  of  what  can  be  done  using  glass  displays. 
Only  a  non-precision  approach  would  move  the  crew  to  select  a  different  mode. 
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VI.  OPEN-ENDED  QUESTIONS  AND  RESPONSES 


A.  INTRODUCTION 


The  intent  of  this  chapter  is  to  take  the  reader  into  the  “details”  of  Continental’s 
B-757  training  program.  The  open-ended  responses  and  summaries  which 
follow  are  an  attempt  to  capture  the  experiences  and  comments,  in  their  own 
words,  of  pilots  progressing  through  their  transition  training,  IOE,  and  eventually, 
flying  the  line.  No  attempt  has  been  made  to  place  judgment  on  the  responses 
with  respect  to  appropriateness,  quality  or  significance,  but  only  to  categorize 
them  for  descriptive  analyses.  The  groupings  and  summaries  are  presented  in 
the  context  of  the  training  program  with  an  emphasis  on  topics  such  as 
problematic  areas,  recommendations  for  improvement,  effectiveness  of  the 
training  aids,  and  the  implications  for  pilots  in  transition  training.  In  this  way,  the 
reader  can  make  his  or  her  own  judgments  on  the  responses. 


B.  METHODOLOGY 


The  challenge  of  collecting  qualitative  data  is  to  reduce  it  into  a  manageable  and 
meaningful  format,  and  then  make  sense  of  it,  especially  when  it  comes  in 
voluminous  amounts.  This  study  was  no  exception,  particularly  when  one 
considers  that  a  set  of  open-ended  questions  were  asked  of  each  pilot  just  after 
their  B-757  training,  and  then  again  after  approximately  one  year  of  flying  the 
line.  This  resulted  in  querying  over  100  pilots  twice  on  the  following  four  topics: 

(1 )  training  for  the  B-757,  (2)  errors  observed  or  committed  on  the  line,  (3)  crew 
coordination  and  procedures,  and  (4)  cockpit  workload.  In  addition,  there  was  a 
question  for  those  who  left  the  B-757  pertaining  to  their  reactions  on  having  left 
the  B-757.  These  five  topics  provided  the  initial  structure  for  presenting  the 
responses. 

Once  the  data  were  organized  and  placed  in  a  coherent  structure,  the  task  of 
identifying  trends  and  regularities  was  proceeded.  There  are  many  ways  to 
identify  regularities  in  qualitative  data  but  the  method  chosen  for  this  study  was 
to  further  characterize  the  responses  by  conducting  an  inductive  analysis.  In  this 
way,  the  data  defined  itself  by  having  the  patterns  and  characteristics  emerge 
out  of  the  chaotic  responses.  This  was  felt  to  be  the  most  appropriate  approach 
rather  than  imposing  structure  upon  the  data,  especially  with  respect  to 
identifying  training  problems  as  well  as  making  recommendations  and  proposing 
intervention  strategies. 


VI  - 1 


As  previously  mentioned,  one  of  the  initial  difficulties  encountered  in  this  study 
was  the  voluminous  amounts  of  open-ended  data  collected.  This  was  further 
compounded  by  the  somewhat  arbitrary  nature  of  some  of  the  responses  and  the 
applicability  to  multiple  categories  (e.g.  cockpit  workload  and  CRM).  Another 
consideration  was  the  longitudinal  nature  of  the  study.  An  attempt  was  made  to 
look  at  each  questionnaire  individually  and  then  summarize  the  topic  as  a  whole. 
We  felt  that  this  approach  would  capture  any  specific  patterns  after  the  initial 
transition  training  in  Questionnaire  2  (Q2)  and  then  once  again  after  flying  the 
line  for  approximately  one  year  in  Questionnaire  3  (Q3).  In  addition,  Q2  and  Q3 
topics  were  summarized  together  at  the  end  of  each  major  question  section. 

Initially,  the  responses  were  transcribed  from  the  individual  questionnaires  into 
an  electronic  format  for  ease  of  manipulation.  The  quotations  are  as  close  to 
verbatim  as  practical,  with  some  minor  editing  such  as  punctuation,  spelling  and 
improvement  to  the  flow  of  the  wording.  Several  responses  contain  editorial 
insertions  by  the  authors  and  are  enclosed  by  these  symbols  <  >.  In  addition, 
exclamation  marks,  question  marks,  and  words  underlined  for  emphasis  are  the 
work  of  the  respondent  and  not  the  authors. 

Once  the  responses  were  electronically  transcribed,  they  were  grouped 
according  to  questionnaire  number  (Q2  or  Q3)  and  placed  in  the  appropriate 
topic  (1)  training,  (2)  error,  (3)  CRM,  (4)  cockpit  workload,  and  for  a  select  few, 

(5)  initial  reaction  having  left  the  B-757.  After  grouping,  an  attempt  was  made  to 
analyze  the  responses  with  a  computerized  narrative  analysis  tool  called 
Quantitative,  Objective,  Representative,  Unambiguous  Modeler  (QUORUM;  see 
McGreevey,  1995,  1996,  1997).  Fora  further  description  of  the  QUORUM 
method  see  Appendix  I.  The  QUORUM  results  on  the  open-ended  responses 
were  inconclusive,  due  to  the  short  length  and  minimal  narration  in  the  pilots’ 
responses. 

Due  to  the  inconclusive  results  of  QUORUM,  a  manual  sort  was  undertaken  to 
develop  the  character  of  the  responses.  The  four  main  topics  were  further 
decomposed  into  the  individual  question  components  as  follows: 

1.0  Training 

Questionnaire  2 

1 .2.1  What  did  you  think  of  your  training  for  the  757? 

1 .2.2  Did  you  have  trouble  with  anything? 

1 .2.3  What  topics  should  receive  more  or  less  emphasis? 

1 .2.4  Please  comment  on  the  training  aids  and  devices. 
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Questionnaire  3 


1.3.1  What  did  you  think  of  the  training  you  received  for  the  757? 

1.3.2  Is  there  any  way  you  would  recommend  it  to  be  changed? 

1.3.3  Did  the  training  program  (including  IOE,  LOFT,  etc.)  prepare  you  to  fly 
the  line? 

2.0  Error 

Questionnaire  2  and  3 

2.1  Describe  in  detail  an  error  which  you  have  made,  or  have  seen  someone 
else  make,  with  the  automation,  that  might  have  led  to  some  undesirable 
consequence.  How  could  it  have  been  avoided?  (equipment  design,  training, 
CRM,  procedures?) 

3.0  CRM 


Questionnaire  2  and  3 

3.1  What  can  you  say  about  crew  coordination  and  procedures  in  the  757? 

3.2  In  what  way  are  they  different  from  previous  planes  you  have  flown? 

3.3  What  areas  can  use  improvement? 

4.0  Workload 


Questionnaire  2  and  3 

4.1  How  would  you  compare  the  overall  workload  in  the  757  compared  to  your 
previous  plane? 

4.2  Please  mention  anything  that  you  feel  should  be  changed  to  help  you 
manage  workload  (procedures,  ATC,  training,  etc?). 

5.0  Departed  the  757 

Questionnaire  3  (only) 

5.1  After  you  left  the  757  and  went  to  another  aircraft,  what  was  your  reaction? 

5.2  What  did  you  miss  about  the  757  avionics  and  automation? 

5.3  What  did  you  like  better  about  the  older  technology  planes? 

5.4  Plane  and  seat  you  went  to:  Aircraft _  Seat _ 

Responses  to  each  question  were  entered  into  tables  according  to  a  major 
keyword  in  the  response  (e.g.  for  error:  “procedures”).  Some  responses  required 
another  entry  according  to  a  minor  keyword  in  the  response,  if  applicable  (e.g. 
“procedures”  AND  “not  following”).  With  this  “keyword”  method,  it  was  a  matter 
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of  cutting  and  pasting  the  responses  into  the  appropriate  categories  and  then 
observing  the  patterns  which  emerged. 


C.  OPEN-ENDED  RESPONSES 


Introduction 


Each  open-ended  response  topic  (1 )  training,  (2)  error,  (3)  CRM,  (4)  workload, 
and  (5)  departed  the  B-757  (if  applicable),  was  examined  individually  according 
to  the  questionnaire  number  (Q2  or  Q3)  and  then  again,  in  a  combination  of  both 
questionnaires  (Q2  and  Q3).  Since  the  authors  wish  not  to  burden  the  readers 
by  presenting  all  of  the  comments  and  responses  received,  only  those  comments 
which  are  typical,  contrary,  or  unusual  in  nature  will  be  presented.  However,  all 
of  the  comments  and  responses  are  available  for  qualified  researchers  by 
contacting  the  authors. 

1.0  Training 


1 .2.1  and  1 .3.1  What  did  you  think  of  your  training  for  the  757? 

Questionnaire  2 

Of  the  84  pilots  who  specifically  stated  their  reaction  to  their  training  experience, 
the  overwhelming  majority  stated  that  the  training  program  was  excellent.  In 
addition,  there  was  no  mention  of  dissatisfied  trainees  nor  any  dislike  of  the 
program. 

•  Excellent/best  training  I’ve  had/outstanding/great.  (45)* 

•  Good/very  good/effective.  (29) 

•  Adequate.  (4) 

•  Inconsistent.  (6) 

‘Number  in  parenthesis  indicates  similar  responses.  If  none  are  present,  then 
the  reader  may  assume  only  one  response  of  that  type. 

Questionnaire  3 

Again,  of  the  62  pilots  who  specifically  responded  to  the  question,  the  vast 
majority  felt  the  training  program  was  good  or  excellent. 

•  Excellent/best  training  I’ve  had/outstanding/great.  (39) 

•  Good/very  good/effective.  (20) 

•  Adequate.  (3) 

•  Inconsistent.  (0) 
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Q2  and  Q3  Summary 


It  is  a  commendable  achievement  for  Continental’s  training  department  to 
receive  such  high  accolades  for  their  B-757  training  program.  There  was  not  a 
single  pilot  who  stated  that  he  was  dissatisfied  or  felt  the  program  was  inferior, 
which  is  in  contrast  to  prior  reports  on  automation  and  training  programs  (see 
Wiener  1989,  BASI  1998).  Even  after  approximately  a  year  on  the  line,  the  pilots 
were  exceedingly  satisfied  with  their  training  program. 

1 .2.2  Did  you  have  trouble  with  anything? 

Questionnaire  2 

There  were  few  direct  responses  to  this  question.  Most  pilots  addressed  what 
should  be  improved  or  which  topics  needed  more  emphasis.  The  following  three 
characteristics  emerged  from  those  who  responded: 

Felt  rushed,  intimidated,  or  uncomfortable  (8) 

•  I  felt  rushed.  (5) 

•  I  felt  uncomfortable  the  whole  ground  school. 

•  There  was  so  much  material  in  so  short  of  time  that  I  am  reviewing  my 
manuals  to  re-leam  all  that  I  missed.  Note:  re-leaming  is  different  than 
reviewing. 

•  Difficult  and  intimidating. 

Autoflight  mode  confusion  (3) 

•  I  had  trouble  adjusting  to  the  use  of  different  auto  flight  modes  and 
some  confusion  as  to  which  button  to  push  and  which  mode  to  use  for 
different  aspects  of  flight. 

•  VNAV  path  is  an  area  that  rarely  operates  as  I  think  it  should,  probably 
because  I  do  not  fully  understand  what  it  is  using  to  make  its 
decisions. 

•  I  didn’t  understand  a  few  things  initially  with  LNAV  and  VNAV,  but 
didn’t  quite  know  just  what  I  should  know  or  ask. 

Oral  exam  (2) 

•  I  had  problems  preparing  for  my  oral  exam.  (2) 

No  problems  or  troubles  (so  stated).  (9) 
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Questionnaire  3 


Once  again,  there  were  few  direct  responses  to  this  question.  Of  those  who 
responded,  the  topics  where  as  follows: 

Felt  rushed  or  intimidated  (6) 

•  The  company  tried  to  squeeze  a  lot  of  new  aircraft,  new  technology, 
and  procedures  in  to  too  short  of  a  time  span  for  a  rating  ride. 

•  Fast  paced  and  intense. 

•  Very  difficult  and  frustrating.  Too  much,  too  soon,  and  too  fast. 

•  Too  rushed. 

•  Felt  intimidated  by  the  automation. 

•  I  have  not  received  “training”  only  what  is  required  to  fill  the  FAA 
requirements  for  training. 

LOFT  and  IOE  (2) 

•  Most  problems  during  IOE  and  LOFT  seemed  to  occur  with  the  pilots 
who  did  not  read  the  manual. 

•  My  only  LOFT  was  an  Atlantic  crossing,  which  I  have  never  done  in  an 
aircraft.  I  would  have  benefited  from  a  domestic  LOFT. 

Q2  and  Q3  summary 

As  previously  stated,  few  pilots  responded  directly  to  this  question.  Most  of 
those  who  replied  to  this  question  had  suggestions  or  ideas  to  improve  the 
program  (see  next  section).  Of  those  who  did,  the  feeling  was  that  the  training 
program  was  rushed  and  contained  too  much  information  for  such  a  short  period 
of  time.  This  seems  especially  true  of  those  with  no  previous  glass  experience. 

1 .2.3  What  topics  should  receive  more  or  less  emphasis  or  should  be 
changed? 

Questionnaire  2 

There  were  numerous  pilot  responses  to  this  question. 

Instructors  (4) 

•  Outside  instructors  <non-Continental  personnel  need  to  be  pilots  or 
trained  on  our  standard  procedures. 

•  There  needs  to  be  some  scheduled  time  with  an  instructor  every  day 
just  on  systems. 

•  Instructors  lacked  confidence. 

•  Instructors  applied  pressure  to  learn  procedures. 

“On-the-line”  learning  (5) 

•  Too  much  emphasis  on  OJT  <on  the  job  training>.  (2) 
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•  The  expectation  of  the  training  department  that  minimal  exposure 
received  in  training  should  be  adequate  is  wrong. 

•  I  noticed  that  other  students  with  no  FMC  background  having  a  harder 
time  with  line  operations. 

•  It  <training>  generally  came  together  on  the  line  with  lots  of  practice 
using  the  equipment. 

Instructional  topics  (16) 

•  I  am  still  not  up  to  speed  on  programming  the  FMC.  (2) 

•  I  would  like  more  in  depth  systems  knowledge.  (2) 

•  Windshear  training  verged  on  overkill. 

•  Training  on  the  CDU  was  almost  non-existent. 

•  More  emphasis  should  be  placed  on  the  aircraft  flight  manuals. 

•  The  FBS  was  over  utilized  in  my  case  (5  years  on  the  B-737-300). 

•  A  more  in  depth  explanation  of  the  IRS’s  function  could  have  been  a 
help. 

•  The  training  was  lacking  nuts  and  bolts. 

•  Instruction  in  Long  Range  Navigation  was  too  deep,  the  experienced 
pilots  knew  better  and  the  domestic  pilots  were  “in  shock”  -  teach  the 
basics  and  keep  it  simple! 

•  There  needs  to  be  a  greater  emphasis  on  CRM  and  the  greater  need 
for  the  crew  to  interact  with  the  automation  (FMC)  and  each  other  to 
preclude  mistakes. 

•  Human-automation  interface  training  would  have  been  more 
meaningful  to  me  if  it  had  been  given  after  the  sim  training  instead  of 
before  ground  school. 

•  Exposure  to  automation  should  be  done  before  training  for  those 
without  prior  experience. 

•  The  FMC  training  needs  to  be  focused  on  “real"  operational  situations. 

•  Being  computer  literate  made  the  FMC  a  breeze  to  understand. 

Questionnaire  3 

Instructors  (5) 

•  Need  a  higher  level  of  experience  on  the  part  of  the  simulator 
instructors. 

•  Good  instructors  and  check  airmen.  They  are  out  to  help  rather  than 
“grade”  the  pilot. 

•  Everyone  involved  was  visible,  available,  and  helpful,  but  certain  once 
the  program  is  fully  integrated  into  our  IAH  facility,  that  will 
unfortunately  change. 

•  The  captains,  simulators,  and  instructor  teaching  was  excellent. 
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“On-the-line”  Learning  (5) 

•  Training  gave  us  the  push-button  knowledge,  but  flying  the  line  was 
the  teacher.  (3) 

•  For  the  most  part  the  training  prepared  me  for  line  flying,  but  actually 
being  on  the  line  in  everyday  operations  and  utilizing  all  of  the 
information  brings  it  all  together. 

•  More  CDU  training  and  operation  as  it  took  3  months  on  the  line  to  feel 
comfortable  with  the  various  modes  and  automation. 

Instructional  topics  (6) 

•  More  time  with  the  automation. 

•  Specify  strict  procedures  for  only  one  pilot  to  program  the  FMC/MCP 
below  10,000. 

•  More  preparation  for  the  oral. 

•  Need  more  time  spent  on  VNAV  operation  and  profiles. 

•  Too  much  emphasis  on  FMC  programming. 

•  IOE  should  be  after  100  hours  <on  the  line>  especially  if  this  is  your 
first  glass  cockpit. 

Q2  and  Q3  summary 

This  section  is  where  one  starts  to  see  some  divergence  in  pilots’  responses  with 
respect  to  the  FMS  and  the  automation  in  general.  The  FMS  training  seems  to 
be  either  insufficient  or  excessive  in  some  cases.  Whether  this  is  related  to  any 
prior  glass  cockpit  experience  or  not  is  unknown.  However,  some  respondents 
state  that  they  have  had  prior  automation  experiences  with  aircraft  such  as  the 
B-737-300/400.  Of  some  concern  are  the  responses  which  refer  to  “on-the-line” 
or  On-the-Job-Training  (OJT)  and  “outside”  or  non-Continental  (non-CAL) 
instructors  who  were  not  practiced  in  company  SOPs.  These  two  situations,  OJT 
and  non-cal  instructors,  may  leave  a  pilot  with  some  ambiguity  in  certain 
situations  and,  as  a  result,  pilots  may  reinforce  erroneous  actions  or  faulty 
assumptions. 

1.2.4  Please  comment  on  the  training  aids  and  devices 
Questionnaire  2 

Positive  comments  (39) 

•  CBT  is  excellent/very  effective.  (13) 

•  The  CBT  allowed  students  to  progress  at  their  own  pace,  and  review 
material.  (9) 

•  Training  aids  were  good/very  good.  (5) 

•  The  CBT  along  with  the  FBS  was  very  exciting/impressive.  (4) 

•  Training  aids  and  devices  were  adequate.  (4) 

•  CBT  and  training  devices  in  a  building  block  approach  is  quite  effective 
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•  The  CBT  was  the  best  I  ever  received. 

•  The  training  aids  and  devices  were  state  of  the  art. 

•  The  CBT  was  better  for  not  having  to  listen  to  an  instructor  ramble  or 
go  off  on  a  tangent. 

Negative  Comments  (58) 

•  We  need  a  FMC/CDU  training  aid  for  practice.  (28) 

•  The  CBT  aids  were  inconsistent  and/or  had  errors.  (17) 

•  One  needs  to  be  able  to  go  directly  to  a  specific  item  instead  of 
listening  to  a  large  portion  of  a  system  to  answer  one  question.  (3) 

•  I  did  not  like  the  CBT,  very  impersonal  and  boring.  (3) 

•  There  needs  to  be  more  questions  and  answers  on  the  CBT. 

•  The  CBT  is  not  the  best  way  to  leam  an  airplane. 

•  CBT  was  slow  and  frustrating  to  use. 

•  The  CBT  training  is  linear  oriented  and  does  not  encompass  the 
complete  scope  of  the  automated  systems. 

•  At  times,  I  had  to  “figure  out”  what  the  computer  answer  was  rather 
than  the  system  comprehension  understanding  in  order  to  progress, 
which  is  negative  learning. 

•  The  CBT  lulled  one  into  a  false  sense  of  confidence. 

•  The  FBS  should  not  be  used  as  a  substitute  for  a  real  sim. 


Questionnaire  3 

Positive  Comments  (5) 

•  The  CBT  was  excellent.  (3) 

•  CBT,  FBS,  and  then  the  full  sim  was  a  logical  progression. 

•  The  CBT  was  the  best  of  my  career. 

Negative  Comments  (19) 

•  There  needs  to  be  an  operating  CDU/FMC  trainer.  (13) 

•  The  CBT  was  distracting  in  several  areas  because  it  had  errors.  (2) 

•  The  CBT  could  be  improved  to  allow  for  more  realistic  FMC 
operations. 

•  CBT  (in  lieu  of  instructor-led  ground  school)  is  sheer  drudgery.  I’m 
sure  that  my  retention  of  systems  and  overall  understanding  of 
systems  operations  is  significantly  lower  on  the  B-757  due  to  CBT. 

•  Very  dull  in  the  personal  computer  trainer. 

•  The  fixed-base  simulator  looked  like  an  expensive  make-work 
government  project.  It  is  not  a  simulator,  but  treated  as  a  simulator, 
and  graded  as  one,  to  the  detriment  of  the  student. 
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Q2  and  Q3  summary 

Many  of  the  pilots  responded  that  the  CBT  was  a  good  or  excellent  training 
device.  However,  some  pilots  mentioned  the  CBT  contained  errors  or  that  they 
had  experienced  frustration  by  having  to  retrace  their  steps  in  order  to  review  a 
particular  topic.  In  addition,  many  pilots  mentioned  a  desire  for  a  workable  FMS 
trainer  which  would  enable  them  to  practice  building  FMC  programming  skills 
and  techniques  outside  of  the  simulators  (fixed  base  and  full  motion). 

These  last  three  issues:  (1)  CBT  errors,  (2)  wading  through  prior  CBT  material  in 
order  to  review  previous  topics,  and  (3)  the  need  for  an  operational  FMS  trainer, 
raise  some  concerns  with  the  authors.  Errors  in  the  CBT  are  inappropriate  for 
pilots  undergoing  transition  training  or  any  other  type  of  airline  training  for  that 
matter.  In  addition,  one  can  empathize  with  a  pilot’s  frustration  by  having  to 
navigate  through  prior  material  in  order  to  review  a  previous  topic  only  one  or  two 
frames  away.  Finally,  we  suspect  the  requests  for  the  FMS  trainer  may  be 
predominantly  from  pilots  without  prior  glass  experience. 

1.3.3  Did  the  training  program  (IOE,  LOFT,  etc.)  prepare  you  to  fly  the  line? 
Questionnaire  3  (only) 

Twenty  eight  pilots  (28)  responded  that  the  training  they  received  adequately 
prepared  them  for  flying  on  the  line.  There  were  no  negative  responses  to  this 
question  and  many  without  a  response  indicated. 

General  comments  on  B-757  training  program 

Questionnaire  2 
Style  of  Instruction 

•  Felt  the  training  was  “bought  cheap”  and  not  kept  up  to  date  with 
changes  or  new  information. 

Curriculum  Development  and  implementation 

•  Coming  off  the  B-737-500/300  made  the  training  easy.  (3) 

•  Coming  from  the  B-737-300/500,  it  seemed  more  like  transition 
training. 

•  I  came  from  the  B-737-300/500  and  was  bored  with  some  simulator 
sessions  -  I  should  have  had  a  “short  course.” 

•  Previous  experience  on  the  B-737-300/500  made  the  transition 
extremely  easy.  My  only  negative  comment  would  be  the  length  of 
training  seemed  a  little  long. 

•  Coming  from  the  B-737-300/500, 1  had  the  advantage  of  being  familiar 
with  the  glass  cockpit  and  FMS  computer  which  helped  me  a  lot  and 
made  the  B-757  training  much  easier. 
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•  I  was  a  Captain  on  the  B-737  and  went  to  FO  position  on  the  B-757. 
The  change  of  seats  was  more  trouble  than  the  aircraft  change. 

•  For  pilots  that  have  never  had  an  FMS  aircraft  prior  to  the  B-757,  it 
requires  a  lot  of  hands  on  training. 

•  The  transition  from  the  B-727  was  a  quantum  leap. 

•  I  noticed  students  with  no  prior  FMC  background  had  a  more  difficult 
time  with  training. 

Administration  and  Scheduling 

•  The  14  hour  day  needs  to  be  reduced.  4  to  6  hours  training/day  with 
study  time  would  equal  an  8  -10  hour  day. 

•  I  wish  I  could  have  obtained  my  manuals  sooner  for  studying. 

•  Being  paired  with  the  same  FO  throughout  the  training  sessions  was 
helpful.  We  lived  together,  studied  together  and  flew  together.  Big 
benefit. 

•  They  did  not  give  us  enough  (almost  none)  information  about  training 
before-hand. 

•  ETOPS  training  should  be  given  after  IOE. 

Questionnaire  3 

Style  of  instruction 

•  Would  like  to  see  a  group  class. 

•  Bring  back  the  classroom  environment  to  create  the  question/answer 
exchanges  from  other  pilots. 

•  If  the  line  environment  was  as  exciting  as  the  training,  I  would  have 
stayed  on  the  aircraft. 

•  The  LOFT  training  was  invaluable.  It  all  came  together  in  those 
sessions. 

•  We  need  a  LOFT  program  for  training  (had  only  one  after  PC)  and 
more  recurrent  training  (we  have  none). 

•  The  B-757  training  was  a  self-taught  course  with  too  much  verbiage  in 
the  supplemental  training  guide. 

Curriculum  development  and  implementation 

•  Solicit  feedback  from  the  pilots. 

•  I  came  off  the  B-737-300  which  is  also  automated.  I  thought  the 
transition  to  the  B-757  was  easy. 

•  No  complaints  except  a  shorter  course  should  be  offered  for  pilots  who 
transition  from  B-737-300/500. 

•  Training  was  well  standardized  and  positive  in  nature. 

•  The  simulator  and  LOFT  sessions  were  very  good. 
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Administration  and  scheduling 

•  The  <oral>  exam  would  have  been  more  relaxed  somewhat  if  some  of 
the  FMC  work  was  saved  until  after  the  oral. 

•  Providing  the  study  manual  and  flight  manual  before  beginning  training 
to  give  the  pilots  a  chance  to  prepare  ahead  of  time. 

•  I  feel  that  training  someone  on  any  equipment,  then  letting  them  sit  for 
three  months  is  extremely  dangerous  and  stupid.  I  lost  currency  twice 
before  I  logged  100  hours.  Floundering  around  in  an  unfamiliar 
cockpit,  trying  to  take  in  the  finer  points  of  long  range  navigation  and 
skirting  23,000’  mountains  on  the  backside  of  the  clock  is  not  my  idea 
of  a  good  time. 

Training  Summary 

Once  again,  Continental’s  training  department  receives  high  accolades  for  such 
a  positive  response  to  their  B-757  training  program.  Certainly  some  areas  could 
use  improvement,  but  the  majority  of  the  pilots  felt  their  training  gave  them  the 
skills  and  information  necessary  to  fly  the  line.  However,  it  is  in  this  training 
section  that  one  starts  to  see  a  dichotomy  between  those  pilots  with  no  prior 
glass  experience  and  those  with  previous  glass  experience.  This  dichotomy  is 
especially  prevalent  in  section  1.3.3  General  comments  on  B-757  training 
program.  In  these  general  comments,  one  finds  pilots  with  prior  glass 
experience  (mostly  B-737-300/500  aircraft)  commenting  that  the  training  was 
easy  or  in  one  instance,  “boring."  On  the  other  hand,  one  can  sense  some  pilots 
struggling  with  learning  a  whole  new  concept  of  flying  and  learning  the  FMS 
associated  with  glass  cockpits. 


2.0  Error 
Introduction 

The  responses  to  this  question  were  read,  sorted,  and  then  categorized 
according  to  the  type  of  error.  While  many  responses  indicated  that  the  error 
had  been  committed  by  the  respondent,  some  responses  were  instances  were 
the  pilot  observed  an  error  either  from  the  cockpit  or  jump  seat.  Several  of  the 
responses  were  complaints  or  irrelevant  comments  and  these  were  discarded 
from  the  categorization.  In  addition,  an  error  was  placed  in  only  one  category 
with  no  multiple  entries. 

2.1  Describe  in  detail  an  error  which  you  have  made,  or  have  seen 
someone  else  make,  with  the  automation  that  might  have  led  to  some 
undesirable  consequence. 

This  topic  was  handled  differently  from  the  other  open-ended  responses,  in  that 
all  the  responses  from  questionnaire  2  and  3  were  merged  to  derive  the  error 
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topics.  Once  the  errors  were  sorted  and  categorized,  the  responses  were  placed 
back  into  their  respective  questionnaires  (Q2  or  Q3).  A  total  of  12  error  types 
emerged  from  the  response  sort  and  analysis  with  the  following  topics  emerging: 


Error  Tvoe 

Total 

=  (Q2  +  Q3) 

2.1.1  Programming  CDU/MCP 

(incorrect,  incomplete,  neglecting  or  not  cross-checking) 

52 

(29  +  23) 

2.1.2  Energy  management 

(altitude  bust,  not  meeting  speed  or  crossing  restriction) 

24 

(16+  8) 

2.1.3  Automation 

(over-reliance,  surprise,  or  loss  of  situational  awareness) 

23 

(8  +  15) 

2.1.4  Action 

(out  of  sequence,  neglected  or  incorrect) 

19 

(6  +  13) 

2.1.5  LNAV 
(setup/confusion) 

14 

(6  +  8) 

2.1.6  Mode  switching 

(confusion  with  mode  switching  or  current  mode  state) 

14 

(9  +  5) 

2.1.7  Procedures 

(incorrect,  incomplete,  or  neglecting) 

14 

CO 

+ 

oo 

2.1.8  Workload 

(time  for  scan,  distractions,  or  excessive  heads-down) 

13 

(5  +  8) 

2.1.9  VNAV 
(setup/confusion) 

12 

(8  +  4) 

2.1.10  Approach 
(setup/confusion) 

10 

(7  +  3) 

2.1.11  Equipment 

(aircraft  systems  configuration  or  NAV  displays) 

8 

(6  +  2) 

2.1.12  Training 
(negative  transfer) 

6 

CM 

+ 

The  following  pilot  responses  are  typical  of  the  errors  or  incidents  which  were 
either  committed  or  observed.  All  the  error  responses  are  not  included  so  as  not 
to  burden  the  reader  with  repetition.  Any  suggestions  of  how  the  error  could 
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have  been  prevented  (e.g.,  via  equipment  design,  training,  CRM  or  procedures) 
took  precedence  and  appear  in  the  transcribed  responses  below. 

2.1.1  Programming  CDU/MCP  (52) 

Questionnaire  2 

•  Wrong  fixes  entered  into  the  computer.  However,  <errors>  are  easier 
to  see  in  the  glass  cockpit. 

•  A  mistake  that  is  being  made  by  all  in  programming  the  route.  If  you 
are  cleared  EWR  to  LAX  on  Route  006,  and  you  try  to  install  Route 
006,  but  a  message  appears  ’’Route  does  not  exist.”  Instead  of  going 
to  the  “Route  Page”  and  manually  placing  the  route  in,  everyone  is 
trying  Route  001,  Route  002,  Route  003,  etc.  until  they  found  a  route 
that  matches  006. 

•  The  speed  knob  is  often  mistaken  for  the  heading  knob  and  vice 
versa.  On  take-off  and  climb-out  this  can  cause  a  decrease  in 
airspeed  at  a  critical  time,  or  the  start  of  a  turn  when  it  is  not  desired. 
This  is  an  equipment  design  problem;  they  (knobs)  are  too  close  and 
too  similar  in  appearance. 

•  We  loaded  the  FMC  manually,  then  the  CDU  kept  flashing  that  it 
wanted  to  be  loaded  automatically  by  uplink  causing  difficulty  for  the 
flight.  We  were  concerned  we  had  loaded  it  manually  and  didn’t  know 
what  the  consequences  would  be. 

Questionnaire  3 

•  Most  common  error  for  all  pilots  is  not  checking  the  FMA  after 
selections  or  <after>  engaging  a  mode  selected  on  the  MCP.  This 
error  can  be  avoided  by  making  sure  that  what  you  have  selected  is 
enunciated  correctly  on  the  FMA. 

•  Captain  entered  the  holding  course  incorrectly!  He  used  the  radial 
verses  the  inbound  course.  He  was  a  little  weak  on  the  FMC. 

•  Setting  the  wrong  altitude  in  the  window  <MCP  panel>. 

2.1.2  Energy  management  (24) 

Questionnaire  2 

•  Depending  on  the  aircraft  to  ensure  meeting  restrictions  have  twice 
resulted  in  potential  violations. 

•  In  LVL  change  with  31 5  knots  below  1 0,000. 

•  I  have  never  been  involved  in  an  “altitude  bust”  in  23  years  of  aviation. 
I  was  involved  in  one  excursion  and  one  trip  later,  almost  another 
excursion  for  the  same  reason.  The  captain  was  flying  both  times. 


VI  - 14 


He  decided  to  hand  fly  in  VFR  conditions  out  of  a  high  density  airport.  He 
failed  to  brief  his  actions  and  responsibilities.  We  were  advised  several 
times  on  climb-out  of  VFR  traffic.  After  performing  my  duties  inside  the 
aircraft,  I  turned  to  the  outside  to  look  for  traffic  and  failed  to  cross-check 
the  captain  who  I  had  never  flown  with  before,  but  was  an  instructor  and 
at  least  a  check  airman.  I  assumed  he  was  flying  the  airplane. 
Unfortunately,  he  was  “outside”  the  airplane  and  had  such  a  high  rate  of 
ascent  that  he  “busted”  the  altitude  level-off  by  more  than  800’.  My  very 
next  trip  was  with  a  different  captain  but  almost  the  same  scenario  except 
I  called  500’  before  level-off  and  again  with  a  high  rate  of  ascent.  The 
captain  thanked  me  for  the  notification.  There  is  a  very  big  need  for  more 
communication  in  these  high  workload  areas.  I  have  learned  from  these 
mistakes.  I  only  hope  the  captains,  who  set  the  CRM  pace  in  the  cockpit, 
also  learn  from  their  mistakes. 

Questionnaire  3 

•  Relying  on  VNAV  path  to  accomplish  required  altitudes  at  certain 
waypoints.  VNAV  is  improperly  programmed  for  the  B-757  engines 
that  CAL  uses. 

•  Altitude  busts.  This  A/C  is  so  geared  to  smoothness  for  level  off  that  if 
intermediate  altitude  is  quickly  selected,  it  is  time  to  disconnect. 

•  LAX  CIVET  arrival.  I  set  hard  altitudes  for  numerous  step-downs 
<while>  operating  on  LNAV.  Busted  1,000’  below  altitude  at  one  VOR. 
First  time  in  22  years  of  flying  that  I  had  to  file  a  NASA  report. 

2.1.3  Automation  (23) 

Questionnaire  2 

•  We  were  cleared  for  a  visual  approach  to  a  parallel  runway  while  on  a 
base  leg  (FO  flying).  The  FO  should  have  disconnected  the 
automation  and  turned  into  the  runway.  Instead,  he  stayed  on  the 
base  leg  course  and  intercepted  the  ILS  at  an  80  degree  angle.  We 
flew  near  the  approach  course  for  the  parallel  runway.  The  FO  was 
too  dependent  on  the  automation. 

•  When  I  programmed  in  an  approach,  the  path  depicted  for  the  IAF  turn 
was  not  what  I  expected  or  could  believe,  nor  did  I  feel  the  A/C  would 
comply  with  the  descent  profile. 

Questionnaire  3 

•  I  flew  one  flight  with  the  autothrottles  inoperative.  While  encountering 
a  mountain  wave,  airspeed  blew  off  to  a  value  near  top  bug.  We  were 
at  FL  410  and  airplane  could  have  stalled.  The  autopilot  kicked  off  for 
some  reason,  which  is  what  got  our  attention.  Extra  engine  power  of 
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B-757  allowed  A/C  to  recover  without  having  to  trade  altitude  for 
speed.  Dependence  on  the  autothrottle  system  took  away  a  set  of 
flight  parameters  that  I  would  normally  monitor  without  this  kind  of 
system  and  which  I  seldom  pay  attention  <to  anymore>.  My  throttle 
techniques  are  rusty. 

•  After  arriving  at  the  LAX  terminal  area,  ATC,  due  to  excessive  traffic 
and  their  inability  to  deal  with  increased  traffic,  asked  us  to  turn  and 
intercept  the  final  course  for  rwy  24R  in  the  north  complex.  This  would 
have  simply  consisted  of  dialing  in  the  corresponding  ILS  freq  and  land 
on  the  corresponding  and  assigned  rwy.  The  FO  felt  compelled  to 
reprogram  the  FMC  for  the  ILS  approach  to  that  <new>  rwy.  All  crew 
interactions  were  suspended  until  he  accomplished  “the  task"  of 
reprogramming  the  computer.  I  repeatedly  asked  <him>  to  dial  in  the 
ILS  freq  to  the  reassigned  rwy.  He  became  “hypnotized"  and  would 
not  acknowledge  my  requests.  His  concentration  on  “pushing  the  right 
buttons”  caused  a  breakdown  of  crew  communications,  loss  of 
situational  awareness  and  left  me  to  fly  the  A/C,  talk  on  the  radio,  set 
the  flaps,  etc.  I  have  flown  with  this  individual  on  other  A/C  and  his 
behavior,  I  feel,  is  unique  to  this  A/C.  He  seemed  surprised  that  after 
he  came  back  we  were  so  close  to  landing.  Training  must  emphasize 
a  threshold  of  priority  in  order  to  fully  maintain  an  integral  crew  during 
last  minute  changes. 

2.1.4  Action  (19) 

Questionnaire  2 

•  The  one  I  have  to  work  on  is  select  LNAV  after  having  been  on  an 
assigned  heading  after  cleared  direct  to  a  fix.  I’ve  missed  that  on  a 
couple  of  times. 

Questionnaire  3 

•  Multiple  instances  of  failing  to  engage  LNAV  after  programming  a 
direct  track  and  slow  to  recognize  that  the  AC  is  in  HDG  mode. 

•  I  was  given  a  direct  to  a  fix  after  being  given  a  vector.  After  inputting 
the  fix  to  which  I  was  given  the  “direct  to”,  I  failed  to  select  LNAV. 

After  about  5  minutes,  I  noticed  the  A/C  on  the  map  display  was 
deviating  from  the  displayed  route. 

•  Captain  forgot  to  activate  and  execute  a  route  in  the  FMS.  I  did  not 
notice  until  ready  for  take-off.  Could  have  been  avoided  if  one  of  us 
had  cross-checked  the  other’s  work. 
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2.1.5  LNAV  (14) 

Questionnaire  2 

•  Twice,  since  I  have  been  in  the  B-757, 1  have  been  surprised  to  see 
which  lateral  mode  is  <displayed>  in  the  flight  mode  annunciator. 

Questionnaire  3 

•  Entering  a  waypoint  way  too  close  to  or  just  past  the  fix  and  having  the 
aircraft  start  a  360  degree  turn.  <l>  switched  to  heading  select. 

2.1.6  Mode  Switching  (14) 

Questionnaire  2 

•  VNAV  switched  to  FLCH.  I  did  not  realize  it  had  done  so  and  I  was  not 
watching  the  step  downs.  I  was  contacting  the  company  and  expected 
the  automation  not  to  miss  a  beat. 

•  Even  with  my  experience,  it  is  very  easy  to  forget  to  cross-check  that 
the  aircraft  is  in  LNAV  and  not  heading  select  or  VNAV  speed  and  not 
path.  VNAV  is  more  critical  than  LNAV.  An  aural  warning  when  VNAV 
changes  to  SPD  from  path  would  be  nice.  Constant  cross-checking  is 
imperative. 

Questionnaire  3 

•  I  have  seen  multiple  occasions  when  the  VNAV  system  defaults  from 
VNAV  path  to  VNAV  speed.  This  is  so  subtle  that  it  is  many  times 
unobserved.  On  previous  FMC  aircraft  that  I  have  flown  (B-737-300), 
the  FMC  gave  a  message  “unable  path  descent",  which  gave  the  pilot 
much  better  info  compared  to  the  subtle  FMA  change  in  the  B-757. 

2.1.7  Procedures  (14) 

Questionnaire  2 

•  The  opposite  pilot  executing  the  CDU  without  the  knowledge  of  the 
other  pilot. 

•  Checklist  items  missed. 

Questionnaire  3 

•  Not  briefing  a  contingency  approach  for  airport  with  low  visibility  (below 
1,200  RVR).  <Suggestion:>  brief  multiple  approaches  and  do  not 
allow  approach  control  to  give  unrealistic  commands. 
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2.1.8  Workload  (13) 

Questionnaire  2 

•  During  TO  and  LND  phases,  as  well  as  VFR,  few  pilots  look  outside. 
Specifically  when  ATC  assigns  “Turn  left  to...”  the  first  action  should 
be  to  glance  to  the  left  in  the  direction  of  the  intended  turn.  Most  pilots 
reach  for  the  MCP  or  CDU  first. 

Questionnaire  3 

•  <We>  had  a  runway  incursion  at  SFO.  <l>  had  a  new  FO  and  <he 
was>  new  to  the  automated  A/C.  A  chime  with  ACARS  sounded  at  the 
same  time  <a>  clearance  to  cross  28L  and  hold  short  of  28R  <was 
received>.  As  captain,  I  heard  to  position  and  hold  28R.  FO  read 
back  hold  short  and  became  engaged  in  inserting  a  delay  code  in 
ACRS  (which  was  not  working)  while  I  taxied  into  position.  Tower  said 
nothing.  <Another>  A/C  was  on  5  mile  final  and  had  to  go  around 
(NASA  report  filed). 

2.1.9  VNAV  (12) 

Questionnaire  2 

•  In  the  VNAV  descent  mode  the  automation  does  not  control  the 
airspeed  very  well. 

•  I  saw  a  pilot  try  to  use  the  cruise  page  to  initiate  a  descent  in  VNAV 
and  get  confused  because  it  did  not  present  the  expected  information. 

Questionnaire  3 

No  responses. 

2.1.10  Approach  (10) 

Questionnaire  2 

•  ILS  capture  problems  in  Mexico  City  and  Bogota  Colombia. 

•  After  selecting  approach  mode  to  ILS  25L,  LAX  approach  controller 
changed  approach  to  ILS  24R  with  a  heading  intercept.  We  had  not 
practiced  an  approach  change  after  all  three  autopilots  were  engaged. 
We  went  HDG  SEL  with  no  reaction  from  the  aircraft.  It  took  both  of 
us  about  5  seconds  to  finally  disconnect  the  autopilots  and  hand  fly  the 
AC  to  the  other  runway  and  reconfigure  for  the  approach.  Also,  when 
we  were  told  to  change  runways,  we  were  told  to  maintain  4,000’  until 
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intercept,  but  GS  had  been  captured.  If  we  had  not  disconnected  the 
AP,  we  would  have  busted  the  4,000’  restriction  which  is  not  good!  So, 
I  suggest  more  emphasis  on  runway  changes  after  APP  mode  is 
selected. 

Questionnaire  3 

No  responses. 

2.1.11  Equipment  (8) 

Questionnaire  2 

•  Problems  with  several  different  pilots  having  confusion  with  heading  up 
verses  track  up  (map  mode). 

•  I  accidentally  turned  on  the  APU  in  heavy  rain  on  short  final  because 
the  two  switches  (APU  and  wiper)  are  identical  and  too  close  together. 

•  I  arrived  at  the  runway  with  the  flaps  up  (for  take  off).  The 
<unextended>  flaps  <were>  discovered  in  the  take-off  checklist. 

•  Almost  missed  a  change  in  flap  settings.  We  use  20  degrees  99%  of 
the  time,  and  grow  accustomed  to  that.  When  it’s  changed  it’s  easy  to 
miss,  especially  if  late  for  departure.  I  finally  caught  it  when  setting  the 
V-speeds  because  they  were  higher  than  normal  (flaps  15  were  called 
for  in  this  instance).  I  have  started  circling  critical  items  in  red  on  the 
ACCU-LOAD. 

Questionnaire  3 

No  responses. 

2.1.12  Training  (6) 

Questionnaire  2 

•  Current  CDU  design  and  or  database  needs  modifications.  In  NAV 
DATA  on  the  B-757  you  can  not  build  your  own  waypoint.  Unlike  the 
B-737,  this  is  a  great  draw  back  if  the  waypoints  you  want  are  not  in 
the  database.  Most  mistakes  I  see  on  the  line  are  in  reference  to  this 
one  item. 

•  FO  tried  to  fly  a  VOR  approach  using  <the>  localizer  function.  He 
reverted  to  VOR/LOC  logic  of  his  older  aircraft. 

•  Old  computer  habits  are  hard  to  break. 

Questionnaire  3 

No  responses. 
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Error  Summary 

The  results  of  the  open-ended  error  question  reveals  that  programming  the  FMC 
and  MCP  seem  to  predominate  the  pilots’  responses.  This  is  noteworthy 
especially  when  one  considers  that  there  were  relatively  little  reduction  in 
programming  errors  overtime  (i.e.  approximately  25%  of  the  population  were  still 
making  programming  errors  after  a  year  on  the  line).  One  would  expect 
numerous  errors  after  transitioning  to  the  line  with  a  tapering  off  of  errors  over 
time.  This  does  not  appear  to  be  the  case  with  this  population  in  that  the 
programming  error  rate  appears  to  remain  steady  over  time. 

Another  interesting  trend  is  the  increase  in  automation  error  types  while  flying  the 
line.  One  would  expect  automation  surprises  after  transitioning  the  line,  and  this 
seems  to  be  the  case,  but  an  interesting  trend  is  the  shift  in  automation  surprise 
responses  in  Q2  to  the  reliance  on  automation  responses  in  Q3.  This  echoes 
the  previously  mentioned  problem  “over  use  of  automation”  in  some  aspects  of 
flight  (see  error  type  2.1.3  Automation  Q3  on  page  16  -  "After  arriving  at  LAX 
terminal  area,..."  were  a  pilot  became  “hypnotized”  by  the  FMC  in  a  last  minute 
runway  change). 

One  area  of  concern  is  the  failure  of  energy  management  during  descents. 

There  are  numerous  responses  regarding  the  failure  to  meet  crossing 
restrictions,  excessive  speeds  below  10,000,  and  altitude  busts.  Previous 
studies  (Wiener,  1989)  have  also  noted  a  high  frequency  of  altitude  “busts"  and  a 
failure  to  meeting  crossing  restrictions  and  this  seems  to  be  the  case  in  this 
study  as  well.  Several  pilots  attributed  the  failure  to  meet  crossing  restrictions  to 
the  “clean”  nature  of  the  B-757.  Meanwhile,  other  pilots  mentioned  ATC’s 
tendency  to  keep  the  aircraft  high  until  the  last  minute  and  then  expect  a  rapid 
descent.  In  either  case,  the  fact  remains  that  the  aircraft  is  not  meeting  the 
speed,  altitude  or  crossing  restrictions  required  of  certain  descents  (and  ascents 
in  some  instances). 

Another  interesting  trend  in  the  error  responses  is  the  increase  in  the  action  error 
types  from  Q2  to  Q3.  The  increase  in  this  category  is  almost  exclusively  failing 
to  select  LNAV  after  being  in  heading  select.  This  is  a  curious  trend  in  that  there 
are  relatively  few  mentions  of  failing  to  activate  other  functions  with  the  FMC  in  a 
timely  manner. 

Other  than  the  previously  mentioned  topics,  the  remaining  error  rates  declined 
significantly  from  Q2  to  Q3  which  would  be  expected  as  the  pilots  gain  more 
operational  experience  on  the  line  and  familiarize  themselves  with  the  aircraft. 
But  this  is  mentioned  with  a  caveat,  in  that  "learning  it  on  the  line”  may  be 
associated  with  its  own  set  of  problems  and  errors  types. 
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3.0  CRM 


Introduction 

As  with  Wiener’s  (1989)  previous  glass  cockpit  study,  pilots  tended  to  view  CRM 
as  a  workload  issue.  In  addition,  the  proliferation  of  automated  aircraft  and  two 
member  cockpit  crews  is  premised  on  the  “communication  of  information”  in 
order  to  maintain  situational  awareness  in  the  cockpit.  This  awareness  is 
particularly  critical  during  busy  flight  regimes  or  when  the  other  crew  member  is 
busy  handling  other  duties  and  is  “out  of  the  loop.” 

Once  again,  the  CRM  open-ended  responses  are  treated  in  a  questionnaire 
specific  manner  with  responses  grouped  according  to  either  Q2  or  Q3  and  the 
summary  a  reflection  of  both  questionnaires. 

3.1  What  can  you  say  about  crew  coordination  and  procedures  in  the  757? 
Questionnaire  2 

There  were  24  specific  responses  to  this  open-ended  question  with  the  following 
being  typical  of  those  who  replied: 

•  CRM  needs  to  be  emphasized  all  the  time  in  the  B-757.  (4) 

•  The  need  for  teamwork/CRM  is  very  important  in  the  B-757.  (2) 

•  The  design  and  layout  of  the  flight  deck  make  crew  coordination  very 
easy  and  effective.  (2) 

•  With  the  abundance  of  information  available  in  the  glass  cockpit,  it  is 
probably  more  difficult  for  pilots  to  ascertain  the  situational  awareness 
of  the  other  pilot  crew  member.  This  places  a  little  more  pressure  on 
the  captain  to  communicate  without  trampling  egos. 

•  As  long  as  everyone  is  operating  on  the  same  page,  then  monitoring  is 
good,  but  when  new  info  is  entered  on  different  pages,  especially  at 
lower  altitudes,  workload  increases. 

•  Crew  members  must  interact  verbally  on  what  modes  are  being  utilized 
and  what  the  aircraft’s  expectations  should  be. 

•  Both  pilots  must  define  workload  sharing  in  the  advance  of  flight  (capt. 
briefing)  in  order  not  to  duplicate  jobs. 

•  The  B-757  is  a  high  performance  airplane  and  things  happen  very 
quickly  in  a  short  amount  of  time.  I  think  it’s  important  to  brief  the  very 
basics  in  the  event  of  communication  break  downs  or  emergencies. 
There  is  a  need  to  brief  the  aircraft’s  automation  and  treat  it  as  a  third 
crew  member. 

•  I  like  the  ways  we  are  operating  and  conducting  procedures  in  the 
B-757. 

•  Excellent  procedures  in  the  B-757.  Keeps  you  outside. 
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Questionnaire  3 


Twenty-six  pilots  responded,  with  the  following  being  typical  : 

•  CRM/procedures  in  the  B-757  are  good/very  good.  (10) 

•  There  is  a  need  to  be  more  of  a  manager  of  the  automated  assets  and 
how  to  use  them.  (2) 

•  The  (B-757)  requires  that  we  brief  each  other  on  what  has  been 
loaded  into  the  FMC.  (2) 

•  Standardization  on  the  B-757  is  good.  (2) 

•  It’s  difficult  to  see  what  the  other  pilot  is  inputting  when  he/she  is 
pushing  buttons. 

•  Coordination  in  the  B-757  requires  “buy-in”  by  both  pilots  or  results  in 
the  need  for  one  pilot  to  maintain  extra  vigilance  if  the  other  doesn’t 
understand  the  system  and  is  unwilling  to  express  it.  The  procedures 
are  designed  well,  but  complacency  sets  in. 

•  CAL’s  B-757  CRM  and  training  procedures  are  the  most  advanced. 

•  Pilots  need  to  treat  he  automation  as  a  member  of  the  team. 

•  One  good  thing  is  the  sophisticated  FMC  is  almost  always  right.  Crew 
coordination  is  pretty  simple  if  you  both  follow  SOPs. 

3.2  In  what  way  are  they  different  from  previous  planes  you  have  flown? 

There  were  four  sub-topics  that  emerged  from  the  response  analysis  with  the 
following  three  topics  being  representative  of  the  responses:  (1)  crew  size 
comparisons,  (2)  procedural  and  checklist  differences,  and  (3)  CRM. 

3.2.1  Crew  size  comparisons 

Questionnaire  2 

There  were  10  replies  comparing  three-crew  member  cockpits  with  two-crew 
member  ones  with  the  following  being  typical  responses: 

•  I  have  always  been  on  a  3-man  crew  and  find  both  of  us  doing 
something  with  the  FMC. 

•  I  have  been  on  a  3-man  crew  for  the  better  part  of  my  time.  Switching 
to  a  2-man  crew  changed  the  way  flight  was  conducted.  Once  you  get 
used  to  not  having  the  third  man  onboard,  the  smoother  flight 
progresses. 

•  Roles  and  pecking  orders  are  much  more  apparent  in  3-man  cockpits. 

•  With  an  automated  cockpit,  each  pilot  must  know  what  the  other  one  is 
doing  at  all  times.  In  previous  planes  (3-man),  I  paid  more  attention  to 
basics.  Including  looking  outside.  Now  both  pilots  just  sit  back  and 
monitor  what  the  airplane  does. 

•  Coming  from  a  3-man  cockpit  (B-727),  covering  all  the  bases  in  a 
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2-man  crew  can  be  a  little  busy. 

Questionnaire  3 

There  were  seven  replies  which  refer  to  crew  size  comparisons,  the  following 

being  typical: 

•  Most  captains,  myself  included,  came  off  a  3-man  crew.  That  took 
some  getting  used  to. 

•  I  believe  the  3-man  crew  is  a  much  safer  operation.  Our  CRM  and 
procedures  are  excellent,  but  situational  awareness  is  unavoidably 
diminished  with  malfunctions  and/or  abnormal  situations  develop. 

•  I  always  have  flown  3-man  AC.  This  was  my  first  2-man  AC,  and  it 
gets  busy.  But,  crew  coordination  is  the  same  to  me. 

•  I  prefer  the  2-man  crew  when  it  works  as  we  trained. 

•  There  is  more  coordination  required  in  a  2-man  crew.  All  my  other 
equipment  has  been  3-man  crews. 

3.2.2  Procedures,  checklists,  and  CRM  comparisons 

Questionnaire  2 

There  were  28  similar  response  with  the  following  being  typical: 

•  No  taxi  checklist.  (2) 

•  Average  CRM.  (2) 

•  Cockpit  flow  as  <compared>  to  specific  checklists.  I  prefer  checklists. 

•  The  B-727  has  a  very  long  drawn  out  series  of  checklists  with  a  lot  of 
switches  and  buttons,  where  the  B-757  checklists  are  short  and 
concise  with  system  checks  being  short  and  quick  as  well. 

•  The  simplicity  of  checklists. 

•  I  came  from  the  B-737-300/500  and  there  is  more  emphasis  on  “hitting 
the  box.” 

•  Each  pilot  has  to  be  aware  of  the  other  pilot’s  input  into  the  FMC  and 
other  pilot’s  thought  process. 

•  CRM  and  procedures  are  for  the  most  part  SOP.  The  main  difference 
is  higher  pre-flight  workload  and  closer  coordination  for  in-flight 
programming. 

•  Crew  coordination  is  about  the  same. 

•  Better  coordination  and  procedures  than  any  plane  I’ve  flown. 

•  <l>  came  from  the  A-300  and  CRM  is  adequate. 

•  CRM  works  very  well  and  is  stressed  more  than  other  planes. 
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Questionnaire  3 


There  were  16  similar  responses  referring  to  checklists,  CRM,  and  procedures: 

•  I  prefer  the  short  checklists  and  flows.  (2) 

•  Checklists  are  simplified  and  there  is  no  taxi  check.  (2) 

•  Same  procedures.  (2) 

•  I  had  previously  spent  9  years  on  the  B-737-300/500.  The  transition  to 
the  B-757  was  fairly  easy.  The  biggest  improvement  was  shortened 
checklists  and  cockpit  layout  (most  noticeably  the  HSI  mode  selector). 

•  Crew  coordination  is  much  more  important  in  the  B-757  than  any 
current  aircraft  in  the  fleet. 

•  There  is  more  coordination  required  on  the  B-757. 

•  CRM  is  stressed  by  CAL. 

•  They  (procedures)  don’t  seem  that  much  different  than  the  CRM 
procedures  for  other  airplanes.  It  is  possible  for  the  PNF  to  get  out  of 
the  loop. 

•  Crew  coordination  is  better  (in  the  B-757). 

•  B-757  crews  seems  to  work  more  as  a  team.  Still  have  “hot-shots” 
who  are  always  on  the  computer  and  push  buttons  too  fast. 

•  The  (B-757)  requires  better  communication  between  crew  members  as 
info  can  be  loaded  in  the  CDU  without  the  other  crew  member  knowing 
about  it. 

•  More  CRM  in  the  B-757,  some  of  it  in  relation  to  the  increased  capacity 
of  the  airplane. 

•  Crew  coordination  is  simpler  because  it’s  easier. 

3.4  What  areas  can  use  improvement? 

Questionnaire  2 

There  were  34  recommendations  made  by  the  pilots  on  how  to  improve 
procedures,  checklists,  and  CRM.  The  typical  recommendation  responses  were 
as  follows: 

•  We  need  a  clearer  separation  of  CDU/MCP  duties.  (4) 

•  More  emphasis  on  standard  procedure/checklist  usage.  (4) 

•  Would  like  a  taxi  checklist  with  the  flight  controls  check  done  at  that 
time.  (2) 

•  There  needs  to  be  additional  care  taken  in  observing  and  cross¬ 
checking  the  programming  of  the  FMC.  (2) 

•  Have  the  PF  make  the  CDU  entries  like  on  the  B-737-300/500.  Why 
not  have  consistency  across  all  aircraft  types?  (2) 


VI  -24 


•  Procedures  for  setting  the  ALT  ALT/warning  on  the  MCP  should  be 
changed.  I  think  that  if  the  PNF  always  made  the  ALT  change  (like  A- 
300),  there  would  be  less  room  for  error. 

•  Would  like  to  see  some  reference  to  flaps  on  a  checklist,  possibly  on 
the  after  start  check. 

•  I  still  find  some  people  who  are  resisting  procedures  to  call  for 
CDU/FMC/MCP  functions  when  they  are  the  pilot  flying,  especially  in 
the  terminal  environment. 

•  Duties  regarding  who  should  program  the  FMC  when  PF  or  PNF. 

•  Most  FOs  do  not  verbally  announce  changes  in  the  MCP  altitude 
settings.  The  flight  manual  mandates  doing  so  and  I  think  this  is  a 
training  or  awareness  item. 

•  I  am  all  for  short  checklists,  but  it  bothers  me  that  our  take-off  and 
landing  checklists  leave  things  to  be  done  after  the  checklist  is 
complete.  We  should  be  at  our  final  flap  setting  before  calling  for  the 
landing  checklist  instead  of  doing  the  checklist  at  20  degrees. 

•  The  company  has  gone  overboard  to  keep  checklists  short.  The  flaps 
not  being  on  the  after  start  checklist  is  a  crime  given  that  there  is  no 
taxi  checklist.  They  are  part  of  the  “flow”,  but  don’t  appear  on  a 
checklist  until  before  takeoff,  when  it’s  a  little  too  late  to  be  lowering 
the  forgotten  flaps.  I’ve  gone  all  the  way  to  the  runway  with  the  flaps 
up,  neither  pilot  noticing. 

•  It’s  hard  communicating  with  all  the  flight  attendants  (we  usually 
carry  5). 

•  Works  fine  as  is,  no  improvements  needed.  (3) 

Questionnaire  3 

There  were  19  suggestions  from  pilots  with  the  following  being  typical  responses: 

•  A  more  specific  defined  policy  of  one  pilot  programming  the  FMC.  (3) 

•  Need  to  have  better  CDU  input  verbalization,  especially  in  the 
terminal.  (2) 

•  More  checklist  discipline:  calling,  responding,  and  timely  execution.  (2) 

•  More  emphasis  on  less  FMC/CDU  below  10,000. 

•  The  approach  checklist  is  used  at  a  time  when  the  cockpit  workload  is 
very  high. 

•  There  is  a  need  for  PNF  to  monitor  FMC  inputs  by  PF,  in  order  to 
verify  and  backup.  There  is  a  tendency  by  some  captains  who  are 
PNF  to  make  FMC  inputs.  This  can  be  confusing  to  the  PF. 

•  I  am  quite  alarmed  that  while  the  B-757  procedures  are  instructed 
during  training,  there  are  many  cross-over  procedures  that  creep  in  to 
line  operations  “well,  that’s  the  way  we  always  did  it  on  the  Airbus.”  We 
need  enforcement  of  the  concept  that  this  is  the  B-757. 
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•  Below  18,000’,  I’d  like  to  have  both  crew  members  in  the  loop  and  not 
talking  on  the  radio  to  company  and  doing  maintenance  write-ups. 

•  I  believe  that  calling  for  the  flight  guidance  changes  helps  keep  the 
non-flying  pilot  “in-the-loop.”  Yet,  there  seems  to  be  tremendous 
reluctance  to  do  so.  It  embarrasses  some  pilots  to  verbalize 
commands  they  are  used  to  performing  themselves. 

•  There  needs  to  be  an  improvement  in  interpersonal  skills.  Some  pilots 
can  not  relate  to  their  fellow  man  and  equally  as  poorly  with  a  know-it- 
all  computer. 

3.5  Miscellaneous  CRM  responses 
Questionnaire  2 

In  questionnaire  2  there  were  nine  responses  which  were  not  easily  categorized, 
as  a  result,  they  were  placed  in  this  miscellaneous  response  category: 

•  I  appreciate  the  active  teaching/applications  of  CRM. 

•  CRM  has  always  been  good  at  CAL  in  my  experience. 

•  I  am  a  proponent  of  strong  CRM.  I  stress  it,  set  the  environment  for  it, 
and  it  seems  to  work.  This  requires  an  open,  receptive  and  forgiving 
captain  for  excellent  CRM  to  work. 

•  Complacency  could  become  a  problem  if  everything  is  loaded  properly 
and  things  work  great.  Success  and  ease  brings  complacency. 

•  At  this  carrier,  I  find  CRM  to  be  very  good. 

•  CRM  is  a  very  good  tool  to  improve  cockpit  inter-relationships. 

CRM  summary 

There  appears  to  be  an  underlying  theme  in  the  pilot’s  responses  which  reflects 
a  need  to  effectively  monitor,  communicate  and  manage  information  on  the  flight 
deck.  In  fact,  one  pilot  referred  to  the  automation  as  a  “team  member”  and 
suggested  treating  it  accordingly.  However,  the  abundance  of  information  could 
“swamp”  a  recently  transitioned  pilot  or  present  difficulties  ascertaining  another 
crew-member’s  awareness  of  the  current  flight  regime.  This  was  alluded  to  in 
several  statements  regarding  further  clarification  on  duties  and  procedures  for 
inputs  into  the  FMC/CDU. 

A  topic  which  received  many  comments  was  the  2-person  vs.  3-person  cockpit. 
This  still  seems  to  be  a  prevalent  topic  among  some  pilots  even  though  the  2- 
person  cockpit  has  been  in  service  for  several  decades.  It  appears  that  most 
pilots  transitioning  from  a  3-person  crew  acclimate  to  the  new  situation  fairly 
easily,  however,  they  are  still  a  few  pilots  who  are  opposed  to  the  loss  of  the 
flight  engineer  and,  in  fact,  returned  to  a  3  seat  aircraft  based  solely  on  that  fact 
(see  section  5.0  For  those  who  have  left  the  757). 
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As  for  procedures  and  checklists,  most  pilots  preferred  the  shortened  checklists 
and  flow  patterns  associated  with  the  B-757.  However,  there  was  mention  of 
“forgetting”  to  set  the  flaps  before  arriving  at  the  runway  thresh-hold  for  take-off 
and  this  raises  some  concern.  In  general,  most  of  the  pilots  endorsed  the  B-757 
procedures,  checklists  and  Continental’s  strong  CRM  approach  associated  with 
this  aircraft. 


4.0  Workload 


4. 1  How  would  you  compare  the  overall  workload  in  the  757  compared  to 
your  previous  plane? 

Workload  Compared  to  Previous  Plane  Total  =  (Q2  +  Q3) 

4.1.1  Much  less  than  previous  plane 

16  =  6  + 

10 

4.1.2  Less  than  previous  plane 

32  =  15  + 

17 

4.1.3  About  the  same  as  previous  plane 

26  =  13  + 

13 

4.1.4  More  than  previous  plane 

12  =  8  + 

4 

4.1.5  Much  more  than  previous  plane 

7  =  3  + 

4 

4.1.6  Shifted  or  different  from  previous  plane 

3  =  0  + 

3 

General  Comments  on  Workload 

Questionnaire  2 

ATC  and  effect  on  workload 

•  Last  minute  changes  causes  the  workload  to  increase  at  critical 
times.  (3) 

•  ATC  has  a  tendency  to  keep  you  high  expecting  a  rapid  descent  and 
or  speed  reduction  which  is  hard  to  do  in  a  B-757. 

•  Under  normal  conditions  it  is  less.  When  ATC  makes  changes  that 
were  not  programmed  in  the  FMC,  then  it  becomes  more  workload. 

•  ATC  calls  at  inopportune  times. 

•  Need  to  allow  for  published  FMS  arrivals.  Too  often  ATC  cancels 
them. 

•  ATC  is  always  changing  to  their  benefit  and  increases  our  workload. 

•  It’s  helpful  when  ATC  understands  how  the  automation  works  and 
clears  us  with  that  in  mind. 

•  ATC  is  inconsistent. 

•  ATC  interferes  with  the  automation  too  often. 
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•  Once  ATC  changes  the  flight  plan  the  FMC  needs  to  be  changed  and 
this  increases  workload. 

•  Some  controllers  at  ATC  still  seem  unaware  of  the  aircraft’s/crew’s 
capabilities  and  do  not  make  full  use  of  what  we  can  do.  Sometimes 
they  even  become  argumentative  when  we  try  <to>  help. 

Phase  of  Flight  and  Workload 

•  The  majority  of  workload  on  the  B-757  is  prior  to  departure  and  after 
landing. 

•  Pre-  and  post-flight  workload  is  higher.  In  flight  workload  is  lighter. 

•  The  overall  workload  seems  to  greatly  reduced  during  most  phases. 
However,  the  loading  of  the  flight  computer  seems  to  take  a  lot  of  time 
and  concentration.  I  am  referring  to  the  pre-flight  phase. 

•  Overall  workload  in  pre-departure,  post  arrival  and  most  terminal 
environments  is  considerable  higher.  Especially  in  a  fluid  ATC 
environment. 

•  The  loading  procedure  during  preflight  is  laborious  and  time 
consuming.  To  do  an  effective  loading  of  the  FMC  takes  a  good  10  - 
15  minutes  with  numerous  opportunities  for  errors. 

•  The  workload  is  far  greater  during  the  departure  and  arrival  phases  of 
flight  for  FMS  aircraft  than  for  non-FMS  ones. 

•  <Workload  is>  much  greater  except  at  cruise. 

•  The  workload  at  the  start  of  the  flight  (in  chocks)  is  the  highest  I’ve 
experienced.  However,  once  airborne,  the  load  is  very  low  and  it’s 
easy  to  lapse  into  complacency.  The  need  for  vigilance  is  even  greater 
for  that  reason.  One  note:  CAT  III  approaches  and  autoland  are 
probably  the  place  where  the  difference  in  workload  is  the  most 
pronounced. 

•  Higher  preflight  <workload>  is  bad  in  B-757. 

•  An  increased  workload  just  after  take  off.  After  that,  much  easier. 

•  The  important  thing,  however,  is  when  it  <workload>  is  less.  I  do  most 
of  my  work  before  the  flight  even  begins.  Then  I  am  able  to  better 
watch  and/or  manage  the  balance  of  the  flight. 

•  I  felt  that  one  area  of  concern  that  I  had,  even  after  IOE,  was  descent 
management.  In  fact,  I  see  the  same  thing  in  many  of  the  new  pilots 
coming  on  the  line.  The  problem  I  am  referring  to  is  not  “planning”  a 
descent,  but  executing  it  safely  and  efficiently  with  the  flight  guidance. 

•  Descents  are  unpredictable  in  the  B-757  system. 

Workload  and  long  haul  operations 

•  On  B-757  there  is  less  workload  on  all  phases  of  flight  except  actual 
ETOPS  and  long-range  NAV  (look  at  ETOPS  checklist). 

•  Very  high!  A  very  high  workload  is  especially  prevalent  during 
international  operations. 
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•  I  just  flew  to  Manchester.  The  trip  over  was  fine.  Coming  back  we  had 
everything  changed  en  route  (track,  ETOPS  altn  3  times,  etc.).  Two 
pilots  were  too  busy  to  do  all  of  this.  An  I RO  was  needed  badly.  If  any 
abnormals  would  have  happened,  we  would  have  been  so  busy  that 
"flying  or  monitoring  the  A/C”  would  have  been  tough. 

Workload  and  lack  of  a  flight  engineer 

•  More  workload  due  to  the  absence  of  a  flight  engineer.  The  pilots  have 
to  pick  up  the  company  and  cabin  workload. 

•  Much  greater  - 1  came  from  3  pilot  A/C  to  2  pilot  A/C  but  automation 
lessens  the  workload. 

•  I  was  previously  on  a  3-man  A/C  (DC-IO)and  there  is  no  comparison. 
Work  was  easier  with  the  S/O  doing  his  job. 

•  I  did  not  assume  V*  of  the  S/O’s  workload,  but  from  50%  to  1 00% 
additional.  When  an  emergency  requires  radio  coordination  with 
company  personnel,  one  crew  member’s  workload  increases  many 
fold.  Being  a  single  engine,  single  seat  trained  pilot,  I  am  somewhat 
accustomed  to  the  workload.  However,  when  the  F/O  has  to  confer 
with  the  capt.  for  information  and/or  decisions,  the  capt.  can  become 
overloaded.  A  third  crew  member  would  be  an  enhancement  in  certain 
environments. 

•  Workload  is  slightly  higher,  but  mostly  during  pre-flight  (no  second 
officer),  but  nothing  unmanageable. 

•  Increased  workload  because  of  a  2-man  cockpit. 

•  Workload  is  automated  and  it  helps  cover  a  lot  of  items  that  were 
manual  by  the  second  officer. 

•  The  second  officer  is  helpful  in  monitoring  approaches,  reading 
checklists,  and  obtaining  ATIS  and  company  data. 

Workload  and  comparison  to  previous  plane 

•  B-757  workload  is  about  the  same  as  B-737-300/500.  (3) 

•  The  whole  cockpit  of  the  B-757  is  much  busier  than  the  old  airplane 
(DC-10)  with  a  3-man  crew,  but  that’s  not  the  only  reason.  The 
automation  has  to  be  programmed  and  that  takes  time,  knowledge, 
and  practice.  The  FMC  is  nice  for  planning  -  if  you  have  the  time. 

•  Much  less  than  B-727  compared  to  flying  and  following  clearances. 

•  Easier  than  the  B-737-300. 

•  Workload  on  the  B-757  is  greatly  reduced  versus  the  B-737-300/500 
because  of  our  long  route  segments  versus  the  short  ones  on  the 
B-737. 

•  The  overall  workload  is  certainly  no  more  than  the  MD-80  or  B-727, 
and,  in  most  situations,  considerably  less.  I  think  the  workload  is 
already  easy  to  manage. 

•  I  find  the  workload  slightly  less  than  the  B-737-300/500. 

•  Much  less  than  the  B-727  and  the  B-737  (due  to  the  long  legs). 
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•  The  B-757  is  not  any  busier  than  any  other  A/C  I  have  flown. 

•  Workload  on  the  B-757  is  greater  than  on  the  older  generation  of 
aircraft.  For  example,  on  the  DC-9,  in  order  to  plan  a  descent,  I  would 
figure  the  distance  needed  to  cross  a  point  at  a  certain  altitude  and  the 
rate  of  descent  or  airspeed  would  be  established  and  both  monitored. 
On  the  B-757,  I  still  have  to  figure  the  descent,  then  check  the 
computer  and  if  both  agree,  then  start  the  descent  at  the  correct  time 
while  making  sure  it  holds  the  speed,  autothrottles  are  coming  up  to  try 
and  hold  the  speed  but  then  reduce  for  the  descent  rate,  and  the 
green  arc  is  correctly  displaying  the  situation.  In  this  instance,  it’s 
easier  to  fly  the  plane  than  the  “computer.” 

•  It  is  higher  than  the  DC-1 0. 

•  My  previous  plane  was  the  DC-10  which  had  3  pilots  thus  spreading 
the  workload  thinner,  but  I  find  the  B-757  the  easiest  plane  I’ve  ever 
flown  both  in  terms  of  workload  and  flighty  characteristics. 

•  The  B-757  and  B-737-300  are  equal  during  gate  departure,  takeoff, 
etc.,  but  the  B-757  is  significantly  higher  during  descent  due  to  its 
tendency  to  get  fast  and  high  on  descents.  This  causes  much  greater 
use  of  speed  brakes,  mode  switching,  button  punching  of  the 

FMS,  etc. 

•  The  overall  workload  in  the  B-757  has  increased  substantially  in  the 
critical  phases  of  flight,  compared  to  the  previous  A/C  I  flew  (B-747,  B- 
737,  B-727,  DC-10,  and  DC-9/MD-80).  An  emergency  arising  during  a 
critical  phase  of  flight  can  overwhelm  even  a  well  trained  and 
experienced  crew.  Your  average  crew  could  be  overloaded  or 
distracted  with  mush  less.  This  advanced  twin  engine,  two  pilot  aircraft 
has  now  been  flying  for  years  and  has  managed  to  stay  out  of  the 
headlines,  perhaps  due  to  its  relatively  small  numbers. 

Workload  and  gaining  experience  with  the  B-757 

•  The  workload  is  now  becoming  less  as  I  gain  more  experience. 

•  Right  now  with  just  a  little  over  100  hours,  the  workload  is  probably 
more  than  my  previous  plane,  but  I  feel  that  with  another  6  months  or 
so,  I  <think>  it  will  be  the  same  if  not  less. 

•  As  I  become  more  familiar  in  the  B-757,  workload  appears  to  be  less 
and  less. 

Availability  of  information  and  situational  awareness 

•  More  information  is  available  to  me  in  the  B-757.  Workload  is  no  more 
or  less,  but  I  have  more  options  and  displays  from  which  to  choose. 

•  The  workload  is  greatly  reduced  in  having  the  ability  to  evaluate  and 
improve  situational  awareness. 

•  Workload  is  much  lower  yet  have  more  information  to  help  make  good 
management  decisions. 
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Questionnaire  3 


ATC  and  effect  on  workload 

•  ATC  has  not  fully  upgraded  equipment  and  is  unable  to  make  full  use  of 
today’s  automation.  It  <B-757>  of  course  is  restricted  by  combination  of 
all  A/C.  Maybe  20-30  years  from  now  when  most  A/C  will  be  auto? 

•  In  high  density  areas,  the  call  in  to  Departure  Control  is  right  at  our 
flap/power  transition  point.  If  ATC  could  live  with  just  the  call  sign 
(company  and  flight  number),  we  could  be  better  at  flying,  looking, 
transitioning  and  being  more  coordinated/safer.  At  EWR  (for  example) 
we  could  auto  switch  to  Departure  Control  at  500  ft  (or  some  agreed  on 
altitude).  The  point  is  that  you’ve  got  radio  calls,  flaps  to  change, 
headings  to  dial,  altitude  to  set  (all  in  5  seconds).  Parcel  it  out. 

•  The  problems  with  ATC,  (LAX  and  SFO)  it’s  hard  to  use  all  the 
automation,  because  airports  like  these  are  too  busy.  Speed  up,  slow 
down  -  stop  your  descent  or  climb  so  you’re  back  to  a  basic  A/C. 

•  ATC  in  arrivals  and  departures  are  the  biggest  workload.  We  need  a 
better  ATC  system  to  manage  ARR  and  DEP. 

•  ATC  procedures  have  become  a  real  pain.  I  don’t  believe  the  changes 
are  necessary  and  I  believe  that  controllers  have  an  agenda  that  they 
are  promoting  through  their  "erratic  changes"  in  traffic  control. 

Phase  of  flight  and  workload 

•  Too  much  company  and  outside  interference  during  block  preflight 
duties  and  check  list. 

•  I  believe  the  workload  to  be  higher  I  the  B-757  during  some  phases  of 
flight  i.e.  preparing  A/C  for  flight  (on  ground)  and  at  altitude  prior  to 
descent  (especially  when  changes  occur  to  expected  clearances,  SIDs, 
runway  changes).  However,  these  normally  do  not  occur  during  critical 
phases  of  flight  -  once  programmed,  the  automated  cockpit  gives  you  so 
much  more  information  during  the  critical  phases  of  flight  i.e.  T.O., 
approach,  landing. 

•  Higher  workload  during  preflight  and  in  general  there  is  less  workload  in 
flight.  However  there  is  an  increase  in  workload  with  the  B-757  in  the 
terminal  area  when  there  is  a  change  in  runway  or  instrument  approach. 
At  high  density  airport  such  as  LAX,  a  greater  EFIS  workload  reduces 
ability  to  look  for  other  airport  traffic.  The  slowness  of  the  FMC 
contributes  greatly  to  the  problem  computer  speed  should  be  increased. 

•  Ground  duties  present  a  much  higher  workload  than  other  equipment 
such  as  B-727,  B-737-1 00/200,  DC-9  &  MD-80  and  close  to  DC-10.  The 
pilots  need  to  arrive  at  the  aircraft  to  set  up  much  earlier  than  the 
Jurassic  jets.  Thirty  min  show  at  operations  at  outstations  is 
unacceptable.  The  duty  day  should  be  1  hour  show  both  at  the  start  of 
the  paring  as  well  as  downline.  Competent  crews  that  are  ahead  of  the 
aircraft  and  not  in  <cabin>  row  23  should  have  no  problem  managing  the 
operations.  If  ATC  can  utilize  FMS  arrivals  and  sequence  other  A/C 
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along  those  routes  radio  transmission  could  be  cut  substantially.  This 
would  provide  a  much  safer  environment. 

•  More  in  chocks,  less  in  taxi,  same  on  T/O,  less  in  climb  and  cruise,  more 
on  descent,  less  on  taxi,  more  in  chocks. 

Workload  and  long  haul  operations 

•  More  workload  on  NATRAC-ETOPS  RTES  <when  compared  to  a 
DC-IO. 

Workload  and  lack  of  a  flight  engineer 

•  It’s  much  higher,  naturally,  since  there  is  one  less  person  in  the  cockpit. 
Normal  arrival  at  gate  communication  procedures  should  be  handled  via 
ACARS  up/down  link.  Having  to  make  several  radio  calls  and  monitor 
frequencies  other  than  TWR/GND  is  inherently  error  prone  in  a  2-man 
cockpit. 

•  Much  greater  workload  on  B-757.  Two-man  cockpit  in  today’s  arena, 
with  all  required  radio  work,  especially  if  something  goes  away,  i.e. 
divert,  emergency,  etc.,  overloads  crew  to  a  dangerous  level.  I  do  not 
have  a  solution,  except  to  bring  back  the  flight  engineer. 

•  I  was  a  F/O  on  B-727.  My  workload  increased  greatly!  2-man  A/C 
should  never  have  been  approved  to  begin  with.  Too  much  inside  and 
not  enough  outside! 

•  <Workload>  is  less  than  B-727  even  with  2-man  vs.  3-man  crew. 

•  Generally  less  (than  the  B-747). 

•  The  previous  A/C  had  a  flight  engineer,  <now>  we  have  the  FMS. 

•  Having  the  flight  engineer  is  a  wonderful  luxury.  However,  in  certain 
environments,  must  notably  terminal  arrivals  and  departures,  a  last 
minute  clearance  change  (change  of  rwy,  route,  or  approach)  can  leave 
me  feeling  overwhelmed.  In  these  cases  I  generally  revert  to  raw  data 
automated  operation  (heading  select,  etc.).  When  time  and  workload 
allows  I  then  bring  the  FMC  up-to-date  again. 

•  Workload  from  a  3-man  aircraft  to  a  2-man  aircraft  increases  no  matter 
what  automation  is  added. 

•  A  third  crew  member  would  be  helpful  to  accomplish  all  “clerical” 
responsibilities. 

•  Much  easier  without  the  F.E.  I  enjoy  doing  the  cockpit  prep  with  all  the 
datalink  hookups  and  not  having  to  forget  to  get  a  word  in  on  a 
congested  frequency.  Captain  can  easily  and  efficiently  validate  the 
accuracy  of  the  data. 

Workload  and  comparisons  to  previous  plane 

•  B-737-300/500  <is>  about  the  same.  (3) 

•  The  initial  loading  of  the  computers  takes  more  time  and  rechecking 
than  the  previous  A/C  (A-300).  The  uplinking  of  data  is  very  helpful 
when  it  works. 
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•  I  feel  the  B-757  workload  is  easier  than  previous  A/C  due  to  glass 
cockpit. 

•  Previous  plane  A-300.  How  about  <a  difference  in>  night  &  day? 

•  I  have  flown  every  airplane  type  in  the  Continental  fleet.  The  B-757  has 
without  a  doubt,  the  lowest  overall  workload. 

•  Much  less  than  B-727  -  once  you  master  <the>  FMC. 

•  The  workload  is  much  less  than  an  MD-80  or  DC-10. 

•  Workload  between  DC-10  and  B-757  is  very  close  to  the  same  -  the  big 
difference  is  that  EFIS  in  the  B-757  results  in  better  situational 
awareness. 

•  Higher  workload  than  B-727  for  arr  -  dep. 

•  The  workload  is  higher  in  the  B-757  than  the  B-737-300  because  of  the 
FMS  design.  Subtle  mode  switches,  like  VNAV  Path  to  VNAV  Speed, 
should  have  never  been  FAA  approved.  This  increases  workload  by 
requiring  a  fixation  on  VNAV  performance  -  to  the  detriment  of 
everything  else  that  is  happening. 

•  Workload?  What  workload?  In  comparison  to  the  MD-80  and  DC-9 
there  is  less  to  do  i.e.  Accuload  via  printer,  PDC  via  printer,  but  A/C 
requires  a  different  approach  and  method  of  conducting  yourself.  They 
also  fly  different  stage  lengths  and  frequencies.  The  B-757  has  to  be 
one  of  the  least  stressful  aircraft,  operating,  usually  one  or  two  legs  per 
day. 

Workload  and  gaining  experience  with  the  B-757 

•  The  first  year  much  heavier.  The  2nd  year  -  the  same;  starting  <the> 

3rd  Year  -  easier. 

•  When  I  first  checked  out  on  the  B-757, 1  found  the  workload  to  be  quite 
high!  I  was  not  accustomed  to  having  to  do  so  many  additional  duties. 

I  now  find  the  work  load  to  be  much  lower.  Automation  helps  and  my 
flow  patterns  are  refined. 

•  Until  recently  the  workload  in  the  B-757  was  about  the  same  as  it  was 
on  previous  aircraft.  The  difference  was  one  of  workload  distribution. 

Availability  of  information  and  situational  awareness 

•  Much  lower  <workload>,  the  automation  allows  you  to  stay  much  further 
ahead  of  the  airplane! 

•  The  workload  on  the  B-757  is  much  less.  The  navigation  tasks  are 
greatly  reduced  and  there  is  a  wealth  of  information  about  the  flight  that 
can  be  accessed.  If  there  is  a  drawback  it  is  that  the  system  is  so  nice  it 
breeds  complacency. 

•  The  map  display  really  makes  the  flight  progress  nicely.  I  like  the 
display  system  with  the  airport  and  runways  shown  on  <the>  screen. 

•  Workload  during  routing  tasks  in  a  low  stress  environment  is  greatly 
reduced.  However,  I  am  not  absolutely  sure  that  the  workload  is 
reduced  during  high  stress  situations,  i.e.  approaches  or  complicated 
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arrival  procedures  particularly  when  a  modification  is  introduced  by  ATC. 
Personally  I  find  the  workload  to  be  increased  because  I  must  decide 
upon  the  manner  in  which  the  change  <takes  place>.  I  may  have 
several  options  available  to  me  and  that  in  itself  complicates  the 
procedure.  With  the  old  “stream  gauges"  you  tuned  it,  dialed  it  and  flew 
it.  Simple.  And  both  pilots  knew  what  procedures  was  taking  place. 

•  Quicker,  more  accurate  assessment  of  overall  situations  as  so  much 
information  is  available. 

•  The  EFIS  helps  my  situational  awareness,  which  makes  the  overall 
workload  on  the  B-757  even  less. 

4.2  Please  mention  anything  you  feel  should  be  changed  to  help  you 
manage  workload  (procedures,  ATC,  training  etc?) 

Questionnaire  2 

•  No  changes  needed  (so  stated).  (3) 

•  It  would  be  nice  if  more  routes  could  be  stored  in  the  database.  (2) 

•  Having  to  reset  the  altitude  to  an  intermediate  altitude  on  the  MCP  on 
a  profile  descent  defeats  the  automation  and  increases  unnecessary 
workload  levels.  The  intermediate  altitude  showing  on  the  moving  map, 
the  legs  page,  and  the  descent  page  are  all  an  adequate  check  that 
the  AC  will  not  bust  an  intermediate  crossing  restriction. 

•  We  should  have  a  sterile  environment  @  20  minutes  prior  to  departure 
and  no  cabin  or  company  contacts  or  duties  after  descending  for  the 
purposes  of  landing. 

•  More  defined  procedures  as  to  capt/F.O.  responsibilities  for  cockpit 
setup  (data  loading,  panel  setup,  etc.).  <Also,>  Orange  County  ATC  - 
very  busy  and  heavy  workload.  ATC  is  calling  at  the  busiest  time 
(tower  to  departure  handoff).  This  occurs  at  the  same  time  as  a  very 
busy  FMS  departure.  Less  talk  would  help. 

•  It  would  help  all  types  of  aircraft  if  the  WX  sequences  would  report 
runways  in  use  at  the  reporting  airport  or  if  we  could  get  ATIS  farther 
away  from  the  destination  (or  alternates  en  route). 

•  ATC  procedures  for  FMS  aircraft  should  be  modified  to  minimize  the 
time  required  for  pilots  to  direct  the  FMC.  This  will  probably  occur 
naturally  as  the  majority  of  airborne  systems  become  FMS 
controlled.  The  problem  exists  due  to  a  mix  of  FMS  and  non-FMS  A/C 
in  today’s  ATC  environment.  Arrival  and  departure  procedures  could 
be  modified  to  simplify  the  phase  of  flight  requirements  for  all  A/C. 

•  The  company  needs  to  standardize  databases  on  both  the  “glass  “  A/C 
(B-737-300/500  and  B-757). 

•  Some  of  the  preflight  CDU  entries  could  be  automated  or  eliminated. 
Examples:  (1)  descent  winds  have  to  be  input  twice,  (2)  engine  oil 
quantity,  (3)  DFDMU  data,  (4)  origin  and  destination  (route  page  1), 
and  (5)  shutdown  fuel. 
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•  No  company  business  below  18,000'  but  call  company  when  on  the 
ground  and  taxing  in. 

•  Complete  and  maximum  utilization  of  datalink  capabilities  in  order  to 
ensure  100%  concentration  in  monitoring  the  aircraft  during  critical 
phases  of  flight  (i.e.,  descent  in  dynamic  traffic  environment,  low 
altitude  transitions,  etc.). 

•  Maybe  an  updated  ATC  system  with  less  power  failures  or  equipment 
problems  would  help  to  bring  down  the  workload  of  both  the  pilots  and 
air  traffic  controllers . 

•  (1 )  Speed  up  the  computer,  and  (2)  an  extension  line  to  ILS  final 
should  not  have  to  be  manually  input.  This  would  eliminate  one  step 
which  is  heads  down  time  in  the  terminal  area. 

•  The  software  should  be  examined  to  see  why  the  airplane  always  gets 
fast  and  high  on  descents.  During  my  first  100  hours,  I  have  only  had 
one  descent  using  VNAV  path  which  did  not  require  a  significant 
application  of  speed  brakes.  This  was  not  the  case  on  the  B-737-300. 

•  The  only  thing  I’d  like  to  see  is  a  redesign  of  ATC  to  cater  to  the 
automated  aircraft  and  not  keep  messing  up  our  plans.  I  believe  they 
are  working  on  it  but  time  will  tell.  After  all,  most  A/C  are  FMC 
equipped  these  days. 

Questionnaire  3 

•  The  slowness  of  the  FMC  contributes  greatly  to  the  problem  <of  high 
workloads  Computer  speed  should  be  increased.  (2) 

•  No  changes  needed  (so  stated).  (2) 

•  Make  it  possible  to  retain  <past>  waypoints  on  the  CRT  for  a  while. 

•  Too  much  company  and  outside  interference  during  block  preflight 
duties  and  check  list. 

•  Reduce  the  flying  workload  for  long  haul  so  more  situational 
awareness  can  be  given  to  terminal  areas. 

•  ATC  should  get  up  to  speed  with  the  rest  of  the  world  with  their 
computers,  training,  facilities,  etc. 

•  I  do  not  have  a  solution  <for  increased  workload>,  except  to  bring  back 
the  flight  engineer. 

•  Approach  checklist  is  a  nuisance  -  should  be  changed. 

•  Training  should  emphasize  MCP  first  below  10,000  ft. 

•  ATC  realizing  and  using  automation. 

•  (1)  Center  the  HSI  display  and  TCAS.  <Also,>  both  now  wipe  out 
anything  under  you  or  immediately  behind  you.  (2)  GPS  updates  to 
position  (both  1  and  2  would  have  helped  the  B-757  crew  in  South 
America  <Cali,  Colombia  accident>). 

•  Automated  ATIS  info  in  ACARS  would  help. 
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•  The  biggest  restraint  I  feel  out  there  is  ATC.  I  calculate  the  optimum 
descent  point  to  the  nano-second  and  ATC  starts  me  down  100  miles 
early,  or  late.  I  don’t  need  an  FMC  for  that. 

•  I  would  change  the  chiming  of  the  no  smoke  sign  at  10,000  ft  to 
18,000  ft.  At  10,000  I  feel  the  crew  is  to  busy  i.e.  slow  to  250kts,  ATC, 
traffic  watch. 

Workload  summary 

As  mentioned  in  the  previous  section  (3.0  CRM),  the  2-pilot  vs.  3-pilot  flight  deck 
is  a  prevalent  topic  among  some  pilots.  Some  of  the  reasons  given  for 
maintaining  the  second  officer  are  for  safety  reasons  (i.e.  “extra”  pair  of  eyes  and 
for  emergency  situations),  but  also  for  routine  duties  such  as  company 
communications  and  paperwork.  In  fact,  one  respondent  mentioned  the  need 
for  a  “secretary”  instead  of  a  flight  engineer  to  handle  “clerical”  duties. 

The  topic  of  ATC  and  workload  received  many  comments  and  were  almost 
unanimously  negative  in  tone.  There  seems  to  be  a  perception  that  ATC  is  a 
major  factor  in  some  high  workload  situations,  especially  in  arrivals,  departures, 
and  failing  to  make  full  use  of  the  B-757’s  automation.  It  is  an  unfortunate 
situation  when  pilot’s  have  a  negative  perception  of  ATC  with  respect  to 
automation,  workload,  and  perceive  ATC  as  acting  in  their  own  interest.  In 
addition,  international  operations  (ETOPS)  also  received  unanimously  negative 
comments  due  to  the  high  workload  involved. 

When  comparing  workload  to  their  previous  planes,  a  majority  of  pilots  perceived 
the  B-757  as  having  less  or  the  same  amount  of  workload  as  before.  There 
were  a  few  comments  stating  that  the  B-757  was  more  workload  than  their  prior 
plane.  However,  one  must  take  into  consideration  whether  or  not  there  was  a 
shift  or  difference  in  workload  when  compared  to  their  prior  plane.  Although  only 
three  pilots  stated  that  there  was  a  shift  or  difference  in  workload,  many  pilots 
commented  that  the  workload  increased  during  pre-departure  and  post-arrival, 
and  decreased  in  cruise.  The  workload  issue  must  also  be  approached  with  a 
caveat  in  that  many  of  the  B-757  flights  were  long-haul  flights  instead  of  shorter 
legs  associated  with  some  glass  aircraft  previously  flown  (B-737-300/500,  etc.). 


5.0  For  those  who  have  left  the  757  (questionnaire  3  only) 

5.1  After  you  left  757  and  went  to  another  aircraft,  what  was  your 

reaction? 

5.2  What  did  you  miss  about  the  757  avionics  and  automation? 

5.3  What  did  you  like  better  about  the  older  technology  plane? 

(Please  state  the  seat  and  plane  you  went  to). 
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•  (Unknown,  unknown).  I  like  the  3-man  cockpit.  Also,  at  Continental, 
the  B-757  trip  schedules  in  the  Houston  base  are  terrible.  For  the 
same  pay,  the  lines  on  the  B-737  are  far  superior,  I  miss  the  B-757 
and  ILS  automation.  But  I  don’t  miss  the  schedules. 

•  (Captain,  MD-80).  I  left  the  B-757  training  program  after  completing 
ground  school  training.  The  avionics  and  automation  surpassed  both 
the  DC-10  and  B-747  I  had  previously  flown  as  first  officer.  A 
company  bid  allowed  me  to  move  to  MD-80  captain,  my  present 
aircraft  position. 

•  (Captain,  B-727).  <l>  will  start  training  on  B-727  March  5,  1996.  <l> 
will  miss  B-757  plane  but  not  the  schedules. 

•  (Captain,  MD-80).  I  felt  it  was  a  step  down  in  technology.  All  the  bells 
and  whistles.  The  glass  cockpit,  the  features  in  the  database.  The 
older  plane  works  better  in  today’s  environment  because  of  the  speed 
of  technology  in  the  A/C  is  exactly  20  years  behind  and  exactly  where 
today’s  ATC  environment  is. 

•  (Captain,  B-737-300).  As  I  stated  earlier,  the  B-737-500  is  more 
advanced  NAV-wise  than  the  B-757.  I  do  miss  some  of  the  system 
automation  and  the  EICAS  system  on  the  B-757  however. 

•  (Captain,  B-727).  The  only  thing  I  miss  from  the  B-757  is  the  map 
mode.  Favorite  items  of  B-727:  the  3-man  crew  of  the  B-727. 

-  ability  to  hand  fly  much  more. 

-  more  “in  the  loop”  -  a  pilot  again  instead  of  systems  monitor. 

•  (Captain,  DC-9).  First  I  went  from  F/O  on  the  B-757  to  F/O  on  the  DC- 
10.  That  wasn’t  too  bad  of  a  transition.  Then  from  DC-10  F/O  to  DC-9 
Captain.  Then  I  saw  the  changes  from  automation.  No  auto  throttle, 
no  full  auto  pilot.  No  VNAV  or  LNAV.  Life  was  good.  Now  I  work. 

•  (Captain,  B-737).  I  have  flown  the  B-737-300  and  B-757  for  10  years. 

I  miss  the  automation. 

•  (Captain,  DC-10).  Moving  to  the  DC-10  was  like  meeting  an  old  friend. 

I  liked  the  B-757  simplicity  of  items  to  be  checked  before  flight  and 
checklists.  I  miss  the  map  on  the  B-757.  The  two  best  features  about 
the  DC-10  are:  three  crew  members  <and>  international  FAR’s 
concerning  flight  time  limitations.  Allows  you  to  have  much  more  time 
off.  The  domestic  trips  on  the  B-757  are  terrible  due  to  ridiculous  FAR 
limitations. 

•  (First  officer,  B-727).  I  bid  off  the  B-757  because  of  the  automation 
and  bad  trips.  The  only  thing  I  really  miss  about  the  B-757  is 
automation  is  the  printer.  The  older  technology  is  real  flying  and  it’s 
fun.  The  older  technology  makes  you  a  better  pilot,  by  hand  flying  and 
using  your  brain. 

•  (Captain,  DC-10).  I  missed  the  moving  map  and  the  auto  departure 
features  in  LNAV  and  VNAV  for  complicated  departure  and  arrivals. 

•  (Captain,  DC-10).  The  B-757  is  the  best  aircraft  I  have  ever  flown. 

The  glass  display  was  easy  to  program  and  maintain  situation 
awareness.  I  have  been  off  the  B-757  for  8  months  and  have  probably 
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forgot  about  small  problems  that  I  encountered.  Navigation 
comparison  of  B-757  vs.  DC-10,  I  hate  the  “old  way”  of  getting  there. 
So  much  more  info  is  quickly  accessible  in  the  B-757;  I  believe  it  to  be 
the  safer  way. 

•  (First  Officer,  DC-10).  Miss  the  FMC  and  map  on  screen.  I  like  the 
3-man  crew  in  DC-10. 

•  (Captain,  DC-10).  The  B-757  instrumentation  results  in  much  superior 
in  situational  awareness  (  a  quick  glance  at  the  Map  and  you  know 
exactly  where  the  landing  runway  is  and  your  orientation  to  it).  The 
DC-10  requires  at  least  two  mental  maneuvers  to  change  the  existing 
instrumentation  into  an  “estimate”  of  where  the  landing  runway  is  -  and 
may  even  require  a  frequency  change  on  a  VOR  or  ILS.  I  changed  the 
A/C  because  of  better  schedules  (more  days  off)  on  the  DC-10. 

•  (First  officer,  DC-10).  I  miss  everything.  I  wish  the  automation  was  in 
the  DC-10  or  B-747-1 00/200.  The  reason  I  left  was  <that>  the 
company’s  workload,  days  off,  and  limited  crew  rest  at  destination  was 
undesirable.  Domestic  flying  is  not  a  challenge  and  boring  so  I 
switched.  My  dream  would  be  a  B-747-400. 

•  (Captain,  DC-10).  I  am  planning  on  returning  to  the  DC-10  ASAP  for 
the  following  reasons:  (1)  3-man  cockpit,  (2)  3-man  cockpit,  and 

(3)  3-man  cockpit.  I  would  very  much  would  like  to  have  “some"  of  the 
technology  to  augment  or  enhance  situational  awareness/auto  land 
capabilities  and  not  to  replace  the  resources  necessary  to  operate 
airlines  safely. 

Summary,  for  those  who  left  the  B-757. 

Seventeen  pilots  responded  to  these  questions  and  most  seemed  to  miss  the 
automation  but  a  few  respondents  preferred  the  older  “steam  gauge”  cockpits 
and  the  three-person  flight  deck.  Several  comments  were  made  regarding  the 
dislike  for  the  B-757  flight  schedules  and  left  the  aircraft  for  this  reason  (ex. 
“bad  schedules”,  “bad  trips”,  etc.).  Of  the  17  respondents,  11  transferred  to  the 
captain’s  seat  in  other  aircraft.  It  is  understandable  to  desire  a  captain’s  position 
and  one  may  suspect  this  may  have  influenced  a  pilot’s  desire  to  leave  the  B- 
757,  especially  if  one  has  never  held  a  pilot  in  command  position  previously. 


D.  CONCLUSION 


In  conclusion,  Continental’s  training  program  for  the  B-757  received  much  praise 
from  the  participants.  This  is  a  commendable  achievement  for  the  training 
department  and  all  those  involved  who  developed  and  implemented  the  program, 
especially  considering  the  B-757  was  a  new  aircraft  for  Continental  at  the  time. 
Although  there  were  numerous  complaints  and  suggestions  put  forth  on  specific 
topics,  not  one  pilot  was  dissatisfied  with  the  training  program  in  general.  This 
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does  not  imply  that  the  training  program  would  not  benefit  from  refinement  or 
incorporating  some  of  the  suggestions  put  forth  by  the  pilots  in  this  chapter. 
Some  of  the  suggestions  and  comments  put  forth  by  the  pilots  are  quite  apropos 
and  worthy  of  further  examination  for  their  potential  applicability  to  CAL’s  training 
program  as  well  as  other  airline’s  training  programs  as  well. 
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VII. 


CREW  RESOURCE  MANAGEMENT  AND 
HUMAN-CENTERED  AUTOMATION  TRAINING 


Our  company  doesn’t  teach  CRM.  We  teach  pilots  to  fly  and  to  manage 
checklists,  communication,  ATC,  and  systems.  CRM  is  an  integrated  part;  the 
parts  are  inseparable.  We  integrate  CRM  into  everything  we  do,  every  minute 
that  the  crew  members  function  together  as  a  crew. 


—  B-757  flight  instructor 


A.  HISTORY  OF  CRM  AT  CONTINENTAL 


Continental  was  early  on  the  scene  of  cockpit  resource  management  (or  “crew" 
resource  management,  which  is  presently  more  in  style  due  to  its  generality).  The 
programs  today,  and  the  general  acceptance  of  CRM  training  can  be  traced  back  to  the 
pioneering  work  of  Captain  Frank  Tullo.  By  the  end  of  the  ‘70s  the  movement  was 
underway  at  several  U.S.  airlines  (Cooper,  White,  and  Lauber,  1979).  By  1981  United 
Airlines  had  its  own  program  (CLR  -  Command,  Leadership,  and  Resource 
Management)  in  position,  and  offered  it  for  sale  to  other  carriers.  As  early  as  1977 
Texas  International  developed  a  simulator  program  that  today  would  be  called  LOFT. 

Tullo  and  others  at  Continental  watched  the  United  experience  with  great  interest.  He 
once  gave  the  following  example  (personal  conversation  with  first  author,  1993)  of  the 
sort  of  thing  that  bothered  him.  A  Boeing  727  taking  off  from  Phoenix  lost  an  engine  at 
Vr.  They  managed  to  get  into  the  air,  and  at  200  AGL  the  crew  ran  the  engine  failure 
checklist.  Tullo  stated  that  the  crew  “got  nothing  in  return”  for  their  efforts.  “They  could 
have  climbed  to  800  feet,  accelerated,  cleaned  up  the  airplane,  and  then  run  the 
checklist.  Perhaps  United  had  the  right  idea:  some  non-techmcal  training  that  would 
emphasize  teamwork  and  communication  skills  might  be  the  answer,”  he  stated. 

Following  the  strike  in  1983  (see  Chapter  I),  Tullo  was  named  lead  flight  instructor  on 
the  727,  and  took  the  opportunity  to  press  for  a  program  in  what  would  later  be  called 
CRM.  The  first  course,  called  Crew  Coordination  Concepts  (CCC),  and  later  called 
Phase  I,  was  something  of  a  hodge  podge  of  personality  and  small  group  dynamics, 
communications,  the  theories  of  the  situation  of  Professor  Lee  Bolman  of  Yale,  and  the 
Blake-Mouton  grid  (see  Foushee  and  Helmreich,  1988,  pp.  201-203).  The  grid  would 
soon  enjoy  great  popularity  in  the  years  to  come.  Continental  management  supported 
the  proposal,  resulting  in  a  two-day  course  which  was  case  oriented,  stressing  as  raw 
materials  carefully  selected  accident  reports.  These  are  the  basis  of  many  of  the 
programs  around  the  world  today. 
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In  1986  the  company  felt  that  the  course  was  running  so  smoothly  that  it  could  be 
marketed  to  other  airlines.  And  in  the  1980's  there  was  a  plethora  of  accidents  that 
would  make  excellent  examples  of  the  need  for  improved  CRM  in  the  cockpit  (see 
chapters  in  Wiener,  Kanki,  and  Helmreich,  1993,  especially  the  chapter  by  Kayten). 
Management  made  a  long-term  commitment  by  appointing  Tullo  as  manager  of  the 
CRM  program.  The  CRM  program  today  runs  under  the  guidance  of  a  committee  of 
pilots,  and  a  full-time  non-pilot  administrator.  In  1988  a  Phase  II  program  (one  day) 
was  installed.  Phase  II  dealt  primarily  with  decision  making.  The  next  (Phase  III) 
program  stressed  teamwork  and  leadership.  Work  began  in  1987  on  recurrent  LOFTs 
for  the  many  fleets  of  Continental.  The  LOFTs  were  strictly  recurrent  exercises,  not 
“CRM  LOFTS”  which  were  gaining  in  popularity,  and  which  might  soon  be  required  in 
order  to  meet  AQP  requirements  (FAA,  1998;  General  Accounting  Office  [GAO],  1997). 
For  an  excellent  discussion  of  LOFT  design,  philosophy,  and  execution  see  R  Butler, 
1993  and  Orlady,  1994. 

Today  the  programs  are  still  influenced  by  Captain  Tullo’s  early  work.  By  1988 
Continental  had  filed  applications  for  AQP  programs  for  most  of  its  fleets.  The  CRM 
programs  were  one  day  for  existing  pilots  and  two  days  for  new  hires.  By  the  1 990's 
most  pilots  in  the  U.S.  had  been  through  some  kind  of  CRM  program.  The  emphasis 
was  changing  from  communications  and  personality  dynamics  to  newer  concepts  such 
as  error  management,  decision  making,  and  situational  awareness.  A  tidal  wave  was 
sweeping  though  airlines’  fleets.  It  was  called  the  “glass  cockpit"  (Wiener,  1989).  CRM 
managers  fell  heir  to  the  unexplored  problem  of  training  for  the  high  technology  aircraft 
(Wiener,  1 993).  The  frustrations  of  the  early  days  of  the  glass  airplane  have  been 
documented  in  Chapter  II.  It  was  concern  over  these  problems  that  brought  the  current 
(NASA/Continental)  study  into  being  For  a  detailed  discussion  of  integrating  CRM  into 
an  AQP  environment,  see  FAA  (1998),  Chapter  9  (Crew  Resource  Management),  and 
McKinna,  (1996a). 


B.  MODERN  APPROACHES  TO  CRM 


The  post-Tullo  era  at  Continental  saw  a  change  of  emphasis.  The  key  word  was 
integrated.  The  quotation  at  the  beginning  of  this  chapter  captures  the  meaning  of  the 
word  when  applied  to  CRM  training.  Integrated  CRM  training  simply  means  that  CRM 
was  no  longer  taught  as  a  subject  matter  in  itself,  a  skill  which  the  airman  would  master 
in  its  abstract  form,  and  might  some  day  have  the  opportunity  to  apply  it  to  his  work. 
Instead  the  subject  matter  would  be  taught  with  CRM  woven  into  each  topic  For 
example,  in  the  B-757  transition  training,  the  instructor  would  discuss  not  only  the 
maneuver  and  the  technical  procedures  to  be  employed,  but  would  include  as  well  the 
callouts  and  briefings  required,  and  show  the  students  how  to  combine  technical  skills 
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with  CRM  skills.  The  communication,  leadership,  division  of  duties,  etc.  would  be 
taught  as  the  ingredients  of  the  maneuver,  not  separate  functions  to  be  performed  by 
the  pilot. 

Influences 


There  were  two  influential  agents  that  led  to  the  move  toward  integration.  The  first 
was  the  demands  of  AQP  (FAA,  1998).  Compliance  with  AQP  required  instructors  to 
teach  integrated  maneuvers  rather  than  procedures,  to  grade  the  CRM  aspects  of  the 
maneuvers  as  strenuously  as  he  might  grade  the  execution  of  procedures,  and  to  grade 
the  crew,  not  individual  crew  members.  This  in  itself  was  a  radical  notion,  that  grades 
would  be  handed  out  to  the  entire  crew,  rather  than  focussing  on  the  performance  on 
individuals  within  the  crew. 


FIGURE  VIM.  Attitude  toward  company  CRM  training. 


The  second  influence  was  the  instructors  that  were  encountered  by  CAL's  initial 
training  cadre  at  Boeing’s  training  center.  Apparently  one  Boeing  instructor  had  a 
tremendous  impact  on  the  Continental  instructors:  he  demonstrated  how  CRM  could  be 
integrated  with  technical  flight  training.  The  training  cadre  was  determined  to  bring 
both  the  spirit  and  the  methods  of  integrated  training  back  to  Houston  to  become 
training  doctrine  at  the  airline.  This  turned  out  to  be  particularly  helpful  in  the  glass 
cockpits  (at  that  time  B-757  and  737-500).  As  one  instructor  put  it,  “The  more  tools  you 
give  the  pilot,  the  more  CRM  is  needed.”  In  previous  field  studies,  Wiener  (1985b, 
1989)  had  said  essentially  the  same  thing,  that  high  technology  cockpits  required  more, 
not  less  CRM  training. 
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The  Success  of  the  Program 


The  success  of  the  CRM  programs  at  Continental  are  shown  in  Figure  VII-1  (Phase  1 ), 
and  in  the  data  from  Phases  2  and  3,  as  displayed  in  Appendix  A,  Page  A-18.  The 
results  of  this  probe  show  an  extremely  positive  attitude  toward  CRM,  compared  to  data 
we  had  seen  in  earlier  opinion  polls  and  attitude  studies.  We  are  not  accustomed  to 
encountering  attitudes  this  favorable.  It  is  difficult  to  pin  down  just  why  the  Continental 
program  in  CRM  has  been  so  favorably  received,  especially  at  a  company  that  had 
recently  been  through  such  brutal  labor  relations.  The  answer  may  lie  in  part  with  the 
prestige  of  the  instructors  responsible  for  designing  and  implementing  the  program.  It 
may  be  also  due  to  the  fact  that  Continental’s  programs  were  always  oriented  toward 
practical  applications  in  the  cockpit,  toward  procedural  compliance,  professionalism  in 
the  cockpit,  and  down-to-earth  presentation  of  accident  reports.  Absent  from 
Continental’s  programs  were  the  “psycho-babble”  and  “parlor  games"  that  alienated 
pilots  in  the  earlier  CRM  programs.  Recently  the  generality  of  CRM  principles  was 
demonstrated  when  they  were  taught  to  non-cockpit  specialists  within  the  company: 
flight  technical  and  maintenance  workers,  flight  attendants,  and  others  (Fotos,  1991). 


C.  HUMAN  CENTERED  AUTOMATION  TRAINING  (H-CAT) 


The  Influence  of  Delta’s  Experience 

In  the  summer  of  1987  Delta  Airlines  suffered  an  embarrassing  series  of  incidents  and 
close  calls.  In  most  cases,  lack  of  communication  or  ineffective  communication 
between  the  crew  members  seemed  to  be  the  problem.  Under  the  direction  of  Captain 
Reuben  Black,  Delta  launched  an  aggressive  program  to  remedy  the  situation  (Byrnes 
and  Black,  1993).  A  CRM  program  was  installed  for  the  entire  pilot  group,  over  9000 
pilots.  New  hires  had  their  own  program  This  occurred  simultaneously  with  a  growing 
concern  over  the  potential  hazards  of  automation  (see  Chapter  I).  One  of  Delta’s 
incidents  involved  the  inadvertent  shutdown  of  both  engines  on  a  B-767  at  about  1000 
feet  after  takeoff  from  Los  Angeles;  but  most  involved  aircraft  with  traditional  cockpits. 
Delta  decided  to  attack  the  automation  issue  at  the  same  time  as  the  communication 
problem.  A  special  automation  task  force  was  formed,  and  out  of  their  deliberations 
came  a  ground  school  course  named  Introduction  to  Aviation  Automation  ('colloquially 
*/.  A.  Squared”). 

Students  about  to  transition  to  a  glass  aircraft  for  the  first  time  went  through  the  course 
It  was  a  model-independent  (“generic”),  non-technical  exploration  of  the  benefits  and 
hazards  of  automation,  making  use  of  a  variety  of  graphical  material,  lectures,  TV  news 
clips,  and  accident  reports.  The  aim  of  the  course  was  to  familiarize  pilots 
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encountering  glass  for  the  first  time  with  the  general  problems  of  operating  a  highly 
automated  aircraft,  and  the  company’s  philosophical  stance.  At  that  time  Delta  flew  the 
following  glass  aircraft:  MD-88,  B-757  and  767.  Within  a  short  time  it  would  add  also 
the  A-310  through  its  acquisition  of  PAA,  and  would  also  obtain  MD-lls.  The 
automation  course  was  considered  by  Delta  to  be  a  great  success,  and  was  soon 
imitated  by  a  number  of  other  carriers  (Domheim,  1996b;  McKinna,  1996a). 

Corporate  Philosophy 

The  term  “philosophy  of  automation”  became  popular,  as  a  result  of  the  work  of  Wiener 
in  automation  (1985b),  and  Degani  and  Wiener  (1994)  in  procedure  design.  The  notion 
was  that  the  highest  level  of  flight  management  should  spell  out  a  philosophy  of 
automation,  an  over-arching  view  of  the  company  and  how  it  would  expect  automation 
to  be  addressed.  From  this  over-arching  view  would  follow,  according  to  Degani  and 
Wiener,  policies,  then  procedures,  and  the  results  would  be  practices  (what  is  actually 
done  on  the  line).  They  called  this  approach  “The  Four  P’s”.  It  became  the  part  of  the 
curriculum  of  the  various  carriers’  automation  course  --  explaining  the  company’s 
philosophy  of  automation.  See  also  p.  11-22. 

Continental’s  Response 

Continental’s  automation  program  closely  followed  the  Delta  model.  It  was  nicknamed 
the  “glass  class.”  After  several  name  changes  it  stabilized  on  H-CAT  (Human- 
Centered  Automation  Training).  [We  shall  use  the  term  glass  class  in  this  report  to 
mean  generically  any  form  of  classroom-based  automation  training,  at  any  company, 
involving  generic  FMS  and  glass  equipped  cockpits.  This  excludes  technical  courses 
on  particular  models.]  At  Continental,  as  well  as  other  carriers,  the  glass  classes 
played  to  mixed  reviews,  for  a  variety  of  reasons. 

1.  Many  of  the  pilots,  although  encountering  glass  for  the  first  time,  and  clearly  in  a 
position  to  profit  from  the  model-independent  training,  were  impatient  to  get 
moving  into  the  “real  stuff”  (aircraft-specific  systems).  Anything  that  impeded 
this  was  considered  a  waste  of  time. 

2.  Classes  included  (1 )  pilots  who  were  virginal  to  glass  or  FMC;  (2)  former  737- 
300  pilots  who  were  trained  in  FMC  operations,  but  not  glass,  and  (3)  737-500 
pilots  upgrading  to  the  757  (glass  to  glass).  Most  of  the  pilots  with  prior  FMC, 
and  certainly  prior  glass  experience  found  much  of  the  H-CAT  course  redundant. 
Most  reported  that  they  enjoyed  the  discussion  of  the  accidents,  but  felt  that  the 
rest  of  the  material  was  a  waste  of  time. 

3.  There  was  a  lack  of  agreement  about  where  in  the  transition  syllabus  the  class 
should  be  placed.  Some  (including  the  authors  of  this  report)  felt  that  it 


VI I -5 


belonged,  as  traditionally  positioned  by  most  carriers,  at  the  beginning  of  the 
first  day,  for  a  variety  of  reasons,  among  them  the  perceived  need  to  allay  fears 
and  misconceptions,  as  discussed  in  Chapter  II.  Others  in  management  and 
training  felt  that  the  course  was  misplaced  by  being  offered  in  the  early  days  of 
transition,  before  the  trainees  even  knew  what  the  terms  meant.  They  felt  that 
the  instruction  should  be  at  the  very  end  of  the  transition  training,  as  a  capstone 
following  ground  school  and  simulator,  a  kind  of  “pre-lOE  wrap-up.”  Clearly  a 
case  can  be  made  for  this  approach. 

4.  We  heard  at  several  carriers  that  the  glass  class  should  follow  the  systems  oral 
examination  by  the  FAA  (and  designates).  Captains,  we  were  told,  could  think  of 
nothing  but  passing  the  oral,  and  were  not  interested  in  any  instruction  not 
provided  for  that  purpose.  The  statement  applies  equally  to  first  officers,  who 
were  also  type-rated  in  CAL’s  757  program  in  order  to  give  the  airline  flexibility  in 
crewing  its  ETOPS  flights. 

5.  Questions  about  cost  of  the  course  and  who  should  teach  it  were  unavoidable. 
Many  of  the  carriers  began  to  reduce  the  time  allotted  at  one  carrier  it  went  from 
a  day  to  a  half-day  to  two  hours,  even  though  the  original  course  was  considered 
highly  successful.  At  the  same  time,  questions  were  raised  about  the  cost  of  the 
instructors.  Cost  conscious  managers  saw  no  reason  why  CRM  or  glass  classes 
had  to  be  taught  by  line  qualified  pilots,  and  at  some  carriers  the  task  was  turned 
over  to  less  expensive  ground  school  instructors.  This  was  an  offense  to  many 
of  the  pilots,  and  especially  to  those  who  had  been  chosen  to  give  CRM/glass 
class  instruction.  Perhaps  a  non-pilot  could  give  instruction  in  systems.  Aircraft 
systems  work  according  to  a  clear-cut  and  unambiguous  design.  So  they  are  not 
difficult  to  teach.  But  only  a  pilot,  who  had  “been  there  and  done  that”  could 
provide  the  rich  and  hard-earned  experience  needed  to  discuss  actual  line 
operations  and  decisions,  crew  interactions,  and  ambiguous  situations. 

For  example,  it  might  seem  ludicrous  to  have  a  ground  instructor  (non-pilot) 
discuss  the  operational  aspects  of  handling  a  communication  or  decision 
problem  (e.g.  whether  to  divert  due  to  an  ill  passenger,  and  where  to  go.)  Only 
a  pilot,  many  might  say,  could  lead  such  a  discussion.  The  debate  over  who  can 
teach  CRM-type  material  continues,  as  airlines  struggle  to  contain  what  at  many 
has  become  runaway  training  costs. 

6.  It  is  generally  felt  that  within  a  few  years,  the  whole  idea  of  an  introduction  to 
flight-deck  automation  will  be  obsolete,  since  all  pilots  will  have  flown  glass 
aircraft,  even  ab  initio  students  right  out  of  training  academies  (Glines,  1990;  Ott, 
1989;  Johnston,  1993b)  This  may  be  true,  but  in  the  short  term,  perhaps 
another  decade  or  two,  there  will  still  be  the  DC-9  and  the  727  crews  working 
their  way  up  the  seniority  ladder,  for  whom  the  introduction  to  automation 
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training  will  still  be  necessary  and  desirable.  Telfer  (1993)  outlines  a  “human 
factors”  course  for  ab  initio  students. 

One  instructor  pilot  who  we  interviewed  offered  this.  Glass  classes  are  being 
overtaken  by  events  in  the  real  world.  They  are  going  out  of  style,  and  need  to  be 
modernized.  And  you  can  no  longer  tell  the  pilots  the  three  adages  of  the  age  of 
automation: 

1 .  Don’t  be  afraid  of  the  computer. 

2.  Don’t  let  the  computer  take  over. 

3.  Don’t  fly  into  a  mountain  pushing  buttons. 

Despite  its  sometimes  rocky  history,  marked  by  frequent  changes  in  the  course 
materials,  pedagogical  approach,  management  techniques,  and  occasional 
philosophical  reverses,  H-CAT  and  CRM  still  exist  at  Continental,  and  in  various  forms 
at  other  carriers.  As  the  General  Accounting  Office  report  on  CRM  stated,  “regardless 
of  the  form,  the  importance  of  CRM  cannot  be  denied  (1997,  p.  5).” 

The  Politics  of  CRM  Instruction 


CRM  committees  at  most  carriers  we  have  visited  or  worked  with  seem  to  live  in 
suspension,  often  distrusted  and  suspect  by  others  within  the  company.  Perhaps  this  is 
because  they  deal  with  “soft"  systems  and  values,  social  sciences  in  an  industry  that 
admires  engineering.  CRM  committees  may  be  misunderstood  and  not  loved  by  all. 
These  committees  do  valuable  work,  often  at  personal  sacrifice,  and  produce  generally 
worthy  products.  But  they  are  still  subject  to  distrust  and  political  vulnerability.  CRM 
revolutions  and  palace  plots  are  not  unusual  anywhere  in  the  industry. 


D.  THE  FUTURE  OF  CRM  AND  H-CAT 


AQP 

So  far  we  have  said  little  about  AQP  in  this  report.  This  is  because  the  movement  was 
not  highly  active  at  Continental  during  the  period  in  which  we  were  collecting  data  and 
conducting  interviews.  Now,  with  the  massive  Boeing  orders  and  deliveries,  the  fleets 
have  been  working  on  AQP  plans.  In  the  way  of  an  update,  we  note  that  all  models  of 
the  737  at  CAL  now  operate  under  AQP  (-100  and  -200  models  have  disappeared), 
and  the  757  will  be  by  July  1999.  There  is  disagreement  among  the  ranks  at 
Continental  as  to  the  economics  of  AQP.  Some  do  not  believe  it  will  save  money  at  all, 
but  may  still  be  worthwhile  because  of  the  scrutiny  of  one’s  programs  or  plans  and  the 
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self-criticism  it  requires.  Others  point  to  cost-savings  in  a  plan  such  as  the  757.  With  a 
higher  rated  FTD,  757  simulator  time  will  be  reduced.  This  certainly  translates  into 
dollars.  Planning  for  the  777  AQP  proposal  produced  similar  results  -  apparent  time 
and  cost  savings  under  AQP. 

The  savings  expected  from  AQP  are  elusive.  Even  some  of  its  adherents  agree  They 
argue  that  an  AQP  program  is  inherently  expensive,  and  reducing  the  amount  of  time  in 
a  program  does  not  compensate  for  this.  What  AQP  does,  they  state,  is  force  the 
training  department  to  do  a  better  job,  but  not  a  cheaper  one.  We  do  not  take  a  stand 
on  this  -  in  time  there  should  be  figures  to  clarify  this  most  important  point. 

H-CAT 


We  envision  a  future  H-CAT  that  will  be  lean  and  cost-conscious,  and  more  “generic” 
(model  independent)  than  its  predecessors.  Whether  this  instruction  will  eventually 
disappear  from  the  training  plan,  since,  as  we  have  suggested,  every  pilot  will  soon 
have  had  his/her  one-time  inoculation,  is  largely  up  to  the  training  departments.  To  a 
degree  this  question  may  depend  on  the  direction  that  AQP  takes  in  the  years  ahead. 
Pilots  will  migrate  in  and  out  of  seats,  as  fleets  expand  in  the  near  future.  Training 
departments  will  be  heavily  loaded  with  transition  training,  upgrades,  and  new  hires. 
The  expansion  has  already  begun  (Sparaco,  1996;  Ott,  1996;  Proctor,  1998a).  As  of 
this  writing  [summer  1 998],  Continental  had  placed  orders  with  Boeing  for  92  next 
generation  737s,  fifteen  B-767-400s,  and  fourteen  777s.  Fifteen  757s  were  also  due 
from  an  original  purchase  of  45  (Proctor,  1998a). 

New  hires,  be  they  experienced  pilots  or  ab  initios,  will  have  already  been  exposed  to 
one  or  more  forms  of  CRM  instruction.  The  big  problem  will  be  to  standardize  a  pilot 
group,  who  come  from  a  variety  of  automation  and  CRM  backgrounds  and  corporate 
cultures.  If  we  had  to  guess,  we  would  see  the  programs  as  no  more  than  a  half-day, 
discussion-oriented  (rather  than  lecture  oriented),  with  heavy  dependence  on  accident 
reports,  and  other  materials  that  can  be  obtained  inexpensively.  If  taped  lectures  are 
used,  they  will  be  employed  to  motivate  discussion,  rather  than  played  without 
interruption  in  the  manner  of  a  classroom  lecture. 

There  are  two  questions  that  we  would  not  place  a  bet  on.  (1 )  Whether  H-CAT 
programs  will  be  placed  at  the  beginning  or  end  of  the  transition  package  (presumably 
not  elsewhere);  and  (2)  Whether  they  will  be  taught  by  line  pilots  or  by  professional 
ground  instructors  (who  may  or  may  not  be  former  pilots).  Airline  training  departments 
will  have  to  decide  when  it  is  time  to  abandon  H-CAT  instruction,  assuming  that  it  does 
not  become  required  subject  matter.  We  predict  that  in  another  10-15  years,  generic 
H-CAT  and  CRM,  as  training  topics  sui  generis,  will  vanish  and  the  worthwhile  material 
will  be  absorbed  by  the  overall  transition  training  syllabus. 
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CRM 


Clearly  CRM,  in  one  form  or  another,  has  a  bright  future  in  the  aviation  industry, 
although  the  future  form  may  not  be  readily  recognizable  by  today’s  practitioners. 
Already  the  predicted  expansion  beyond  the  cockpit  is  taking  place,  with  CRM  being 
taught,  and  we  presume  applied,  in  such  domains  as  maintenance,  the  cabin,  and 
dispatch  (Fotos,  1991;  Helmreich,  Wiener,  and  Kanki,  1993;  Helmreich  and  Merritt, 
1998;  Proctor,  1998a).  It  is  not  difficult  to  envision  a  company,  airline  or  otherwise, 
where  all  of  the  employees  are  trained  in  CRM  techniques,  and  that  communications 
are  thereby  improved.  For  this  to  work,  we  caution  that  at  least  within  a  company  there 
must  be  consistency  of  philosophy,  approach,  and  instruction.  We  would  not  want  to 
see  one  segment  of  a  company  trained  on  “Type  A"  CRM  and  another  on  “Type  B.”  A 
CRM  culture  clash  within  an  organization  might  be  an  interesting  thing  for  social 
scientists  to  observe,  but  would  not  be  helpful  to  the  organization.  The  predicted 
expansion  of  CRM-like  instruction  into  non-aviation  fields  (Helmreich,  Wiener,  and 
Kanki,  1993)  has  already  come  to  pass.  We  will  not  pursue  that  matter  here,  as  it  is 
beyond  the  scope  of  this  project. 

One  trend  that  we  expect  to  continue  and  expand  is  for  the  airframe  manufacturer  to  be 
the  provider  of  the  CRM  instruction  and  materials.  Airbus  Industrie  has  led  this  field 
CRM  instruction  is  part  of  the  “package”  when  a  carrier  buys  an  Airbus.  “A  pragmatic 
approach  to  automation  is  the  key  element  in  our  training,”  according  to  Captain  Pierre 
Baud,  senior  vice-president  for  training  and  flight  operations  support.  This  approach 
was  the  result  of  the  business  plan  of  that  company,  who  saw  their  products  as  aircraft 
not  only  for  technologically  advanced  nations,  but  for  the  Third  World  as  well.  From  the 
beginning,  Airbus  sought,  in  their  instruction,  to  understand  and  manage  cultural 
affinities  and  differences  (Sparaco,  1996,  p.  133).  As  part  of  its  training  doctrine, 
Airbus  will  use  unusually  small  training  groups,  sometimes  reaching  a  student-to- 
instructor  ratio  as  low  as  1:1 . 

The  trend  toward  manufacturer-provided  instruction  will  no  doubt  continue.  Its  success 
may  depend  on  whether  crew  members  can  accept  instruction  that  is  generic,  rather 
than  model-bound,  generic  not  only  with  respect  to  models,  but  to  company  culture. 

We  can  already  imagine  the  emergence  of  a  joke  about  the  first-day  in  CRM/H-CAT 
class.  The  instructor  begins  his  introductory  remarks  with,  “This  instruction  is  designed 
specifically  for  pilots  flying  the  you-fill-in-the-blank  aircraft  for  you-fill-in-the-blank 
airlines.  The  standardization  of  cockpits  of  various  models,  which  Airbus  has  stressed, 
makes  CRM,  as  well  as  other  instruction,  easier.  There  are  simply  fewer  exceptions 
that  must  be  taught.  Is  keeping  instruction  “generic”  a  good  thing  or  bad?  Due  to  the 
costs  involved,  there  may  be  no  choice.  The  carrier  cannot  expect  to  either  develop  its 
own  tailored,  model-specific  packages,  or  obtain  them  from  the  airframer. 

Finally,  we  predict  the  end  of  the  era  of  “parlor  games",  personality  inventories, 
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authority  grids,  and  psycho-babble.  Perhaps  such  simplistic  approaches  were 
appropriate  in  the  early  days  of  CRM,  since  little  else  had  been  developed  But  they 
have  not  served  us  well.  The  link  between  personality  and  flying  duties, 
communication  or  technical,  still  is  not  established  except  in  the  extreme.  In  brief,  the 
personality  approach  was  not  line-oriented. 

Now,  in  the  age  of  AQP,  the  preparation  of  an  AQP  proposal  to  the  FAA  forces  all  the 
training  issues:  CRM,  H-CAT,  LOFT,  use  of  simulators  and  flight  training  devices,  and 
others.  A  757  training  captain  put  it  this  way.  “We  (each  fleet)  must  make  our  own  AQP 
program,  and  not  try  to  use  an  adaptation  of  another  fleet’s  proposal.” 


E.  IN  CONCLUSION 


AQP  proposals  guide  the  development  of  the  flight  training  programs.  But  they  must 
not  only  answer  to  the  FAA,  but  must  as  well  be  appropriate  to  the  carrier’s  perceived 
mission,  and  consistent  with  the  corporate  culture.  As  Degani  and  Wiener  pointed  out 
(1994),  flight-deck  procedures  for  operating  the  same  piece  of  equipment  vary  from 
company  to  company,  reflecting  the  carrier’s  culture.  Thus  AQP  and  the  training 
doctrine  it  spawns  must  be  sensitive  to  and  flexible  for  local  conditions,  values,  and 
goals.  The  CRM  and  automation  programs  must  do  the  same.  There  is  no  one  correct 
way  to  design  any  training  program,  especially  one  as  culturally  sensitive  as  a  CRM 
class.  Under  AQP,  the  FAA  mandates  a  CRM  program,  but  stops  short  of  telling  the 
carrier  what  to  do  (Orlady,  1994). 

It  is  regrettable  that  there  has  been  such  a  lack  of  creativity  in  addressing  the 
automation  philosophies.  Every  philosophy  statement  we  have  seen  so  far  has  been  a 
rehash  of  Delta’s.  They  usually  follow  this  pattern: 

1 .  These  are  the  advantages  of  automation. 

2  This  is  the  philosophy,  from  which  policies  will  flow. 

3.  The  company  expects  every  pilot  to  be  able  to  use  the  automation  at  every  level. 

4.  But  the  ultimate  decision  on  the  level  chosen  remains  with  the  pilots. 

We  hope  that  in  the  future  some  more  imaginative  work  will  be  done  on  determining 
philosophies  of  automation. 
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VIII.  DISCUSSION 


A  INTRODUCTION 


In  this  chapter  we  shall  briefly  discuss  a  number  of  training  concepts,  based  on  what 
we  have  found  from  the  Continental  experience,  field  observations,  and  a  literature 
review.  These  discussions  are  not  meant  to  be  comprehensive  --  more  complete 
articles  and  book  chapters  are  cited.  We  will  include  in  this  discussion  the 
administration  of  training,  and  the  costs  of  training.  Finally  we  will  take  a  look  at 
training  technologies  of  the  near  future,  and  how  they  affect  the  future  of  transition 
training. 

At  most  airlines,  training  has  been  highly  tradition-based.  Usually  when  a  new  plane  is 
brought  into  the  fleet,  the  training  department  operates  as  it  did  before,  until  a  dramatic 
change  in  technology  (e.g.  the  jet  engine;  the  swept  wing,  computer-graphic  visual 
displays  in  the  simulator;  the  FMS  and  glass  cockpit)  comes  along.  This  calls  for  a 
dramatic  change  in  training,  a  “paradigm  shift.”  Seldom  is  there  a  paradigm  shift  in  the 
training  pedagogy  or  method  --  the  big  changes  are  hardware  driven  (e.g.,  visual 
displays  in  simulators)  or  regulation  driven  (e.g.  AQP). 


B.  COSTS  OF  TRAINING 


For  various  reasons,  training  has  become  a  runaway  cost  at  many  carriers.  Throughout 
this  chapter  we  shall  be  concerned  with  the  rising  costs  of  training,  and  the  acquisition 
and  use  of  various  training  devices.  It  is  difficult  to  find  materials  on  training  costs.  We 
recommend  Dornheim’s  (1992)  article  on  the  MD-11  training  program  at  Douglas.  With 
the  rapid  modernization  of  the  fleets,  airlines  are  scrambling  to  buy  simulators  and 
lease  simulator  time.  They  are  also  acquiring  less  exotic,  though  still  highly  expensive, 
flight  training  devices,  some  of  which  are  so  sophisticated  (and  expensive)  that  they  are 
essentially  simulators,  lacking  only  visual  scenes  and  motion  platforms. 

New  Aircraft 


The  vast  fleets  of  new  aircraft  carry  with  them  the  high  costs  of  transition  training  for  the 
pilots  who  will  fly  them.  Some  of  this  will  be  improved  by  commonality  between  models 
of  the  same  company.  Airbus  Industrie  deserves  honorable  mention  for  their 
contribution  to  commonality:  their  aircraft  have  very  similar  cockpit  designs  and 
configurations,  making  transition  training  less  burdensome,  and  allowing  a  single 
simulator  to  be  superficially  changed  to  allow  its  use  by  different  fleets. 


VIII-1 


For  Boeing,  it  was  more  difficult  to  achieve  commonality  (except  in  the  757/767s  which 
have  essentially  the  same  cockpit  —  and  same  type  rating),  as  they  did  not  have  the 
privilege  of  starting  ab  initio  and  designing  a  fleet  of  aircraft.  The  757/767  was  a 
revolutionary  design,  which  had  no  technological  parents  in  the  civilian  world.  The 
continuation  of  the  Boeing  advanced  aircraft  was  the  747-400,  a  redesign  of  it’s 
traditional  747-100/200/300.  The  -400  had  advanced  EFIS/FMC  systems,  and  a 
two-pilot  flight  deck.  The  777  flight  deck  is  based  on  that  of  the  747-400.  The  new 
generation  737  aircraft  promise  a  high  degree  of  commonality.  The  great  popularity  of 
all  models  of  the  737,  as  well  as  the  new  has  sparked  a  flurry  of  simulator  orders  and 
construction  of  simulator  buildings  and  bays,  and  a  scramble  to  purchase  simulator 
time  from  various  sources. 

Collective  Bargaining 

Collective  bargaining  by  the  pilots’  unions  also  affects  the  price  of  training. 

Increasingly  the  pilot  contracts  contain  negotiated  language  about  the  type  and  amount 
of  training,  the  time  of  day  at  which  it  can  be  conducted,  and  the  administration  of  the 
training.  For  example,  some  contracts  contain  language  making  performance  on  LOFT 
sessions  “non-jeopardy”,  meaning  that  grades  or  other  performance  measures  are  not 
reported  to  management,  but  used  only  as  feedback  by  the  student  and  the  instructor. 

Due  to  the  costs  of  providing  transition  training,  and  the  fact  that  pilots  in  training  are 
essentially  non-productive,  management  attempts  to  limit  the  migration  from  seat  to 
seat,  sometimes  by  putting  a  freeze  (i.e.  2  years)  on  further  bids  after  training  for  a  new 
seat. 

But  enforcing  this  is  often  more  difficult  than  it  appears.  With  rapid  acquisition  of  new 
technology  aircraft,  management  finds  itself  breaking  its  own  rules  by  unlocking  the 
freeze  process  and  allowing  newly  trained  pilots  to  enter  new  bids.  This  was  carried  to 
an  extreme  in  June  1998  when  Continental  and  IACP  agreed  on  a  pilot  contract  that 
changed  the  entire  pay  process  to  the  more  conventional  “weight  differential”  system  -- 
the  bigger  the  airplane,  the  greater  the  pay.  A  discussion  of  the  longevity-based 
system  and  its  effect  on  the  training  facility  can  be  found  on  page  IV-2.  As  we  have 
pointed  out  In  Chapter  IV,  the  change  from  a  longevity-based  pay  scale  to  the  more 
widely  used  weight  differential  scale  resulted  in  a  top  to  bottom  rebidding  for  seats  in 
the  summer  of  1998.  It  was  called  a  “flush  bid”.  The  entire  pilot  assignment  list  was 
abandoned  and  each  pilot  was  free  to  bid  any  seat  at  the  airline.  We  will  not  discuss 
the  effect  of  the  contractual  change  in  detail,  as  it  took  place  beyond  the  data  collection 
phase  of  the  study.  But  we  here  note  that  the  flush  bid  placed  an  enormous  load  on  the 
training  center,  as  any  dramatic  contractual  change  might. 

For  the  757  fleet,  whose  composition  we  discussed  previously,  the  change  was 
dramatic:  the  younger-than-usual,  technology-oriented  captains  and  first  officers  stood 
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to  be  bumped  out  of  the  757  by  senior  pilots  who  had  been  flying  lighter  aircraft. 
Suddenly  the  757  looked  attractive  as  a  mid-weight  aircraft  for  mid-seniority  pilots,  as  it 
was  at  other  airlines  (Wiener,  1989).  The  resurrection  of  the  B-767  order  made  it  even 
more  so.  The  only  thing  that  kept  the  757  from  being  more  attractive  was  its  reputation 
for  poor  schedules  (from  the  pilots’  point  of  view),  and  the  heavy  concentration  of  757 
lines  in  Newark,  which  was  not  considered  a  desirable  area  in  which  to  settle. 

Management  Backing  --  A  Clean  Sheet  of  Paper 

We  have  written  briefly  in  previous  chapters  on  the  importance  of  management’s 
commitment  to  quality  in  aircraft  training  and  its  support  for  new  transition  programs. 
The  phrase  we  heard  time  and  again  from  the  757  flight  management  group  was,  “We 
had  the  right  people  and  a  clean  sheet  of  paper.”  The  meaning  of  this  was  that  the  757 
cadre  was  free  to  propose  the  program  they  felt  was  best,  unfettered  by  tradition  or 
orthodox  ways  of  doing  things.  And  all  important,  they  had  a  commitment  from 
management  that  any  reasonable  request  would  be  supported  financially.  These  were 
important  pledges  on  the  part  of  management:  they  not  only  assured  the  fleet  cadre  of 
support,  but  also  relieved  them  of  budgetary  game  playing  and  financial  uncertainties. 
This  freed  them  to  design  the  program  that  they  felt  was  best  suited  to  the  airline  and  to 
the  task  at  hand,  relieved  of  financial  concerns  and  confident  of  management  backing. 

As  it  turned  out,  there  was  an  added  dividend  for  Continental.  So  successful  was  the 
757  program  that  when  it  became  time  to  prepare  for  the  777,  which  arrived  in 
September  1 998,  that  the  cadre  for  that  fleet  was  drawn  from  the  757.  The  wheel  had 
turned:  the  757  fleet  managers,  who  had  only  three  years  before  been  given  carte 
blanche  to  build  their  team,  found  the  757  ranks  being  raided  for  the  777. 

AQP 


Toward  the  end  of  the  project  we  were  hearing  hints  both  at  Continental  and  elsewhere 
that  carriers  that  had  installed  FAA-approved  AQP  plans  found  that  they  were  not  living 
up  to  its  claims  for  lower  training  costs,  due  primarily  to  the  high  “up-front”  costs  of 
designing  a  program,  obtaining  FAA  approval,  and  then  implementing  and  managing  it. 
This  matter  is  also  discussed  in  the  previous  chapter.  We  are  not  certain  whether  this 
view  is  the  result  of  systematic  studies  of  the  costs  of  AQP,  or  just  opinions.  As  we 
heard  frequently,  AQP  is  “data  intensive”.  And  data  intensive  is  dollar  intensive.  There 
was  general  agreement  that  even  without  cost  reduction  AQP  was  worthwhile,  since  it 
forced  training  management  to  look  very  carefully  at  what  they  were  doing,  and  take  full 
advantage  of  any  opportunity  to  cut  costs.  Others  held  that  AQP  has  performed  “as 
advertised,”  and  has  been  well  worth  the  investment.  Only  time,  and  experience 
training  under  AQP  will  tell  the  story. 
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C.  FUTURE  PILOTS  AND  FUTURE  TRAINING 


Ab  Initio  and  Very  Low-Time  Pilots 

It  is  difficult  to  estimate,  at  any  given  time,  the  market  for  new  airline  pilots.  On  the 
demand  side,  it  varies  with  the  plans  of  the  airlines  to  expand  their  fleets  and  route 
structure.  We  say  expand  because  that  is  the  expectation  for  U.S.  carriers  for  at  least 
the  next  decade  (FAA,  1 991 ).  The  same  is  probably  true  for  other  parts  of  the  world  as 
well,  with  the  possible  exception  of  Asia,  where  major  airlines  are  actually  cutting  back, 
due  to  the  weakening  of  business  prospects.  The  economic  downturn  in  Asia  may 
indeed  be  temporary  phenomenon  and  fleet  development  may  resume,  though  perhaps 
not  at  the  optimistic  levels  of  predictions  made  in  the  mid-1 990s.  Where  will  the  pilots 
for  these  expanded  fleets  come  from? 

The  availability  of  former  military  pilots,  the  traditional  source  of  at  least  one-half  of  the 
airline  crew  members  at  U.S.  carriers,  depends  in  turn  on  the  military  budget  voted  by 
Congress,  which  impacts  two  areas:  (1 )  The  amount  of  the  budget  devoted  to  training 
new  military  pilots;  and  (2)  the  restrictions  on  the  freedom  of  military  pilots  to  leave  the 
service.  Presently  military  flight  training  is  at  a  low  level;  even  graduates  of  the  Air 
Force  Academy  cannot  count  on  a  flying  slot.  About  half  of  the  graduating  class  is 
currently  assigned  to  non-flying  posts. 

The  regional  airlines  are  another  principal  source  for  the  majors,  but  the  same  question 
can  be  raised  about  the  regionals:  where  will  their  pilots  come  from?  Will  the  regionals 
be  able  to  keep  their  young  and  inexperienced  pilots,  will  the  regionals  become  very 
expensive  farm  teams  for  the  majors?  One  should  not  assume  that  “regional"  implies 
low  technology  cockpits.  Many  of  the  regionals  are  now  flying  glass  cockpit  aircraft  fully 
as  automated  and  high  tech  as  those  of  the  majors. 

Another  alternative  is  the  ab  initio  (from  the  beginning)  pilot.  (This  is  discussed  briefly 
on  p.  11-16.  This  student  has  trained,  usually  at  his/her  own  expense,  obtained 
commercial  and  instrument  ratings,  and  joined  the  pool  of  available  pilots  (see 
Johnston,  1993b;  Telfer,  1993;  Marino,  1993).  The  prospect  has  never  been 
particularly  attractive  in  the  U.S.,  perhaps  because  the  pool  of  available  pilots  has 
never  been  that  dry  (see  Glines,  1 990).  CAL  does  not  plan  to  recruit  ab  initio  or  very 
low-time  pilots. 

The  usual  migration  pattern  would  be  to  a  flight  school  to  serve  as  an  instructor  to  build 
up  flying  time,  thence  to  a  regional,  and  hopefully  later  to  a  major  air  carrier.  The  ab 
initio  program  may  be  sponsored  by  the  carriers,  as  it  is  in  parts  of  Europe  and  Asia, 
where  there  is  no  other  source  of  pilots,  and  the  carriers  must  take  a  hand  in  training 
them.  Unlike  the  expected  migration  pattern  of  new,  low-time  pilots,  these  ab  initio 
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pilots  may  go  straight  from  school  to  a  major  carrier.  Their  first  duty  aircraft  could  be  an 
A-320  or  a  new  generation  737.  It  is  difficult  for  most  of  us  to  believe  that  a  300-hour 
pilot  could  handle  the  duties  of  a  first  officer  in  a  highly  technical  aircraft  such  as  an  A- 
320.  But  the  carriers  who  have  tried  it  say  it  works  well.  Fiorino  (1998)  describes  one 
of  those  rare  cases  in  which  a  school  went  on  search  of  bad  weather  in  which  to  train: 

Emirates,  the  international  carrier  of  the  United  Arab  Emirates,  will  be  sending  new 
recruits  for  ab  initio  commercial  airline  pilot  training  to  the  School  of  Aviation  Sciences  at 
Western  Michigan  University  (WMU)  in  Battle  Creek.  Until  now  the  carrier  has  trained 
its  student  pilots  at  British  Aerospace  Flying  College  at  Prestwick,  Scotland.  Aside  from 
the  technical  facilities  and  training,  other  factors  in  Emirates’  decision  to  switch  its  basic 
training  included  the  “more  challenging  weather”... Student  pilots  will  train  for  62  weeks  at 
WMU’s  Pilot  Training  Center,  graduating  with  a  Joint  Aviation  Authorities  commercial 
pilot  license  and  instrument  rating.  Training  will  include  1,000  hours  of  ground  school 
and  300  hours  of  flight  and  simulator  training,  including  3  hours  in  a  fully  aerobatic 
aircraft. 

Children  of  the  Magenta  Line 


Captain  Bruce  Tesmer  of  Continental  (personal  communication,  1998)  speaks  of  the 
new  brand  of  pilots  as  “children  of  the  magenta  line.” 1  His  concern  is  shared  by  many: 
that  these  pilots,  who  go  essentially  straight  from  the  cockpit  of  a  trainer  to  the  cockpit 
of  an  advanced  FMS  aircraft  never  build  the  basic  airmanship  and  sense  of  the 
airplane  that  comes  from  working  one’s  way  up  through  traditional  cockpits  and  finally 
to  the  FMS/glass  cockpit.  An  example  is  described  by  Proctor  (1988):  “Cadets  with  no 
flying  experience  are  sent  overseas  for  about  a  year  to  receive  an  initial  pilot's  license, 
normally  from  flight  training  centers  in  the  United  Kingdom.  Upon  graduation,  they 
spend  a  year  as  flight  engineers  in  Singapore  747s  before  moving  to  the  copilot’s  seat 
in  either  Boeing  757s  or  Airbus  A-310s.”  Thus  the  pilot  has  one  year  of  ab  initio 
training,  a  year  at  which  he/she  does  not  touch  the  controls,  and  then  finds  him/herself 
performing  the  duties  of  copilot  in  a  highly  sophisticated  cockpit. 

This  is  clearly  a  legitimate  worry.  What,  the  critics  ask,  would  happen  if  an  A-320 
captain  were  incapacitated.  The  300+  hour  ab  initio  pilot  would  more  than  have  his/her 
hands  full.  Others  dispute  this,  saying  that  if  such  an  event  occurred,  at  least  the 
inexperienced  pilot  would  have  available  advanced  autopilot  modes  to  help  save  the 
day.  Better  that  they  are  flying  an  A-320  than  a  DC-9.  Only  time  will  tell  on  this 
argument. 


1  The  magenta  line  refers  to  an  executed  course  displayed  on  the  HSI. 
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PC-Based  Training 


We  have  noted  previously  (p.  11-18)  the  possibility  of  making  use  of  the  growing 
popularity  and  versatility  and  declining  price  of  personal  computers.  If  every  pilot  could 
have  his  own  computer  (see  also  pages  IV-6  to  IV-7)  or  at  least  have  one  available, 
then  these  could  be  networked  so  that  instruction  could  be  managed  for  the  pilot  group 
(Nordwall,  1995).  We  envision  a  computer  network  by  which  pilots  could  receive 
instruction  on  their  own  schedule.  They  could  download  and  print  out  portions  of  the 
lesson  if  they  wished,  or  merely  take  instruction  from  the  screen.  For  a  review  of  the 
potential  for  PC-based  training  in  aviation,  see  Koonce  and  Bramble,  1998. 

PC-based  training  has  high  initial  costs,  but  low  per-user  costs.  Tied  to  the  web  (see 
below),  it  would  guarantee  that  all  users  would  have  the  same  version  of  the  program, 
the  latest  version  at  that,  and  would  eliminate  much  of  the  cost  of  printing  and 
distribution  of  materials  compared  to  traditional  training.  It  would  be  easy  for  training 
administration  to  correct  errors  at  the  central  source,  and  to  maintain  centralized 
quality  control.  Dornheim  (1996a)  provides  an  example  of  a  PC-based  trainer  for  FMS 
function.  FMS  training  is  where  it  may  be  needed  the  most,  as  this  is  the  area  that  is 
costly  for  training  using  current  technologies,  and  the  area  where  new  trainees  feel  the 
weakest  in  their  IOE  and  early  line  experience.  A  low-cost,  PC-based  alternative  may 
be  the  answer.  Dornheim  goes  to  suggest  that  one  of  the  biggest  advantages  is  that  a 
$14  million  simulator  is  not  held  up  while  pilots  learn  FMS  functions  and  programming. 
A  commercial  software  house  provides  the  system,  with  a  cost  (at  the  time  of  his 
publication  - 1996)  of  $  200,000  for  a  site  license. 

A  PC-based  network  would  be  useful  for  such  activities  as  learning  new  systems  when 
they  appear  (e.g.  EGPWS),  new  procedures  (e  g.  firefighting  and  security),  and  other 
topics  normally  put  off  until  recurrent  training.  In  fact,  recurrent  training  is  a  good 
example  of  instruction  delivered  at  a  very  high  cost.  Ground  instructors  must  visit  each 
base  time  and  again  until  all  pilots  have  received  the  required  subject  matter,  and 
special  subject  matter  for  the  current  year.  We  believe  that  the  effort  previously  put 
into  lesson  preparation  could  go  into  preparing  software,  allowing  network-wide  use 
and  permanent  storage  of  instructional  material. 

We  would  not  envision  the  network  as  playing  a  major  part  in  transition  to  a  new 
aircraft,  as  we  recommend  in  the  next  chapter  that  an  instructor  be  available  when 
students  are  involved  in  CBT  activities.  The  reader  may  recall  that  many  student  pilots 
in  the  original  cadre  objected  that  at  Boeing  an  instructor  was  not  available  to  answer 
questions  or  clarify  material,  or  in  some  cases  to  confirm  that  the  instruction  contained 
an  error.  In  spite  of  this,  there  would  be  some  material  for  transition  training  that  could 
be  available  on  the  instructional  net,  that  could  reduce  the  heavy  load  on  the  student 
during  ground  school,  a  subject  of  frequent  complaints  (“drinking  from  a  fire  hose”). 
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The  development  of  these  materials  could  impose  a  heavy  burden  on  the  carrier. 
Hopefully  consortia  of  carriers  could  be  formed  that  would  develop  and  share  the 
software.  They  would  have  to  be  extremely  careful  when  the  instruction  contained 
procedures,  rather  than  just  introduction  to  systems,  as  even  for  the  same  hardware 
and  software,  each  carrier  has  its  own  procedures  (Degani  and  Wiener,  1994),  and  this 
would  be  reflected  in  computer  based  instruction.  To  the  degree  that  aircraft  could  be 
standardized  across  companies,  CBT  would  be  potentially  useful  and  also  lower  in 
price.  But  we  do  not  anticipate  a  widespread  movement  toward  cross-carrier 
standardization. 

Another  approach  is  to  have  the  mainframers  play  a  larger  part  in  development  of 
computer-based  (and  other)  material,  just  as  Airbus  Industrie  has  done  with  CRM 
materials.  They  have  been  successful  in  developing  general  CRM  instruction  and  then 
tailoring  it  to  the  culture  of  the  individual  airlines. 

Friendly  and  Unfriendly  Programming  of  the  CBT 

While  attending  the  ground  school,  we  noted  several  examples  where  the  CBT 
programming  was  not  particularly  supportive  of  the  student.  Most  of  these  have  since 
been  corrected,  and  we  will  discuss  programming  and  instructional  design  in  the  next 
chapter.  We  include  these  “unfriendly”  cases  in  this  chapter  only  as  examples  of  what 
can  happen  in  mechanized  instruction. 

1 .  By  and  large  the  CBT  is  a  well-used  and  well-formatted  device.  The  lessons  are 
clear  and  logical  and  the  way  the  lesson  plan  mixes  CBT,  instructor  briefing  (2 
hours  a  day)  and  fixed  base  trainer  (FBT)  is  quite  effective.  Particularly 
beneficial  is  the  manner  in  which  questions/answers  that  are  provided  within  the 
instruction.  Also  helpful  is  the  proficiency  test  at  the  end  of  the  lesson,  and  the 
fact  that  the  student  must  stay  with  a  lesson  until  mastery  on  the  test.  If  the  test 
is  “failed”  there  is  guidance  on  what  areas  were  missed,  and  a  review  can  take 
place.  We  think  an  effective  pedagogy,  due  to:  (1 )  its  ability  to  aid  in  the 
transfer  of  technical  information;  (2)  its  ability  to  focus  on  the  student’s  apparent 
weak  spots;  and  (3)  its  self-motivating  property.  Everyone  likes  to  do  well  on 
tests,  including  self  tests,  even  if  there  is  no  immediate  reward  or  punishment  for 
performance. 

2.  The  CBT  formatting  has  some  flaws,  the  biggest  being  the  way  it  goes  backward 
to  earlier  frames  when  requested  (BACK).  The  problem  is  the  student  has  no 
way  of  knowing  how  far  back  it  is  going  to  go.  Usually  the  trainee  just  wants  to 
hear  the  current  “frame”  again  -  there  is  no  capability  to  do  this.  The  student 
should  be  able  to  go  back  to  beginning  of  current  lesson,  or  merely  replay  the 
current  frame. 
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3.  The  beginning  trainee  should  be  given  a  hard  copy  of  the  acronym  definitions  -  it 
is  distractive  to  have  to  stop  instruction  and  branch  to  the  definitions  page. 

4.  There  is  a  confusing  terminology  in  the  electrical  portion  of  the  program.  When 
a  question  is  missed,  the  tape  says  “the  correct  answer  is  in  blue.”  What  it 
should  say  is  “the  correct  answer  is  in  the  blue  frame.”  Blue  is  used  in  the 
electrical  lesson  to  designate  AC  systems  (DC  in  red),  and  this  results  in  able 
unnecessary  confusion. 

Miscellaneous  Notes  from  Training 

1 .  The  QRH  is  not  in  a  user-friendly  format.  It  is  difficult  to  find  items.  Where 
would  you  look  for  a  wheel  well  fire?  It  is  under  landing  gear,  brakes,  and 
hydraulics.  Maybe  all  fires  should  be  grouped  together.  We  are  not  prepared  to 
say,  so  this  is  not  a  recommendation.  Hot  start  is  in  the  alpha  table  of  contents 
under  “abnormal  starts.”  No  problem  if  you  know  that. 

2.  Ground  instructors  we  have  known  love  instructional  “gimmicks”,  which  they  coin 
and  use  effectively.  One  we  learned  was  the  instructor’s  restaurant  analogy  for 
MCP/FMA  agreement.  This  is  an  all-important  source  of  possible  serious  error, 
entering  something  other  than  what  was  intended  into  the  computer.  The 
instructor  advised  the  students:  think  of  MCP  as  the  waitress  -  you  tell  her  what 
you  want.  But  the  FMA  is  the  table.  The  only  way  to  know  if  you  got  what  you 
ordered  from  the  waitress  is  to  look  at  what’s  on  the  table,  [simple,  neat,  to  the 
point,  and  easy  to  remember], 

Web-Based  Instruction 


The  locus  of  instruction  must  also  be  considered,  with  pilots  living  in  far-flung  places, 
often  commuting  around  the  country  to  their  bases.  In  days  gone  by,  some  instruction 
(such  as  recurrent)  was  made  available  at  each  base,  to  avoid  bringing  thousands  of 
pilots  to  a  training  center.  One  example  where  training  was  carried  out  by  devices  in 
each  base  was  the  introduction  to  TCAS.  Companies  developed  a  variety  of 
instructional  devices,  including  slide  shows  and  VCR  tapes,  to  provide  the  training  for 
the  new  device.  There  may  have  been  computer-based  instruction  for  TCAS  as  well, 
but  we  are  not  aware  of  any. 

But  usually  pilots,  especially  those  living  at  some  distance  from  their  bases,  pass 
through  the  base  very  quickly,  with  little  time  for  instruction.  Clearly  the  answer  is  to 
bring  the  instruction  to  the  pilots’  homes,  so  they  can  do  it  at  their  convenience.  There 
are  a  variety  of  ways  to  do  this,  but  packaging  material  and  sending  it  out  by  whatever 
carrier  has  it  perils  and  limitations.  The  world  wide  web  (WWW)  offers  an  attractive 
alternative:  material  could  be  stored  on  the  web  by  the  training  department,  and  then 
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down-loaded  by  each  pilot  into  his  computer.  (For  an  example,  see  Hughes,  1998, 
p.65).  This  would  bring  new  instructional  material  to  the  doorstep  of  the  pilot  group, 
wherever  they  may  live,  and  do  it  in  a  timely  manner. 

Web-based  instruction  and  delivery  would  allow  virtually  instantaneous  updating  of 
materials,  and  could  ensure  that  all  pilots  have  a  current  version  of  instructional 
materials  and  manuals.  It  could  also  allow  students  to  post  notice  of  what  they  believe 
to  be  errors  in  the  materials,  and  obtain  rapid  corrections  from  flight  training 
management.  Likewise,  questions  could  be  asked  by  the  students  by  posting  on  the 
web.  It  would  be  management’s  responsibility  to  monitor  the  web  for  questions, 
comments,  and  presumed  errors,  and  respond  appropriately. 

In  the  next  chapter  we  will  discuss  conclusions  and  recommendations  for  training  for 
the  high-tech  aircraft.  Many  of  our  recommendations  will  involve  the  training 
techniques  discussed  in  this  chapter,  and  may  be  somewhat  repetitious  of  this  writings 
in  this  chapter. 
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IX.  CONCLUSIONS  AND  RECOMMENDATIONS 


In  this  chapter  we  propose  some  guidelines  for  transition  training  and 
administration,  and  for  appropriate  use  of  automated  equipment,  with  the 
emphasis  is  on  training  issues.  Our  conclusions  are  based  on  our  work  at 
Continental,  as  well  as  visits  to  other  airline  training  centers,  and  jumpseat 
experience  with  several  carriers.  The  conclusions  and  recommendations  may 
apply  more  to  one  airline  than  another,  depending  on  their  corporate  culture, 
present  state  of  training,  fleet,  type  of  operations,  and  many  more  factors.  We 
have  tried  to  be  cost-conscious  in  this  chapter,  remaining  mindful  of  the  price 
tags  as  well  as  the  benefits  of  our  recommendations. 

A  Generalization 


“Training  must  be  considered  during  the  design  of  all  cockpit  systems  and 
should  reflect  that  design  in  practice.  Particular  care  should  be  given  to 
documenting  automated  systems  in  such  a  way  that  pilots  will  be  able  to 
understand  clearly  how  they  operate  and  how  they  can  best  be  exploited,  as  well 
as  howto  operate  them.”  (Billings,  1996,  pp.  11-11,  11-12). 

Management  Support 

When  initiating  a  new  program,  particularly  a  large  program  such  as  transition  to 
a  newly  acquired  aircraft,  management  should  find  the  best  person  for  the  job, 
and  then  give  him/her  a  “clean  sheet  of  paper”,  meaning  put  aside  the  past,  and 
launch  the  program  with  an  open  mind  and  the  full  support  of  management. 

(See  also  p.  VII 1-3.)  Management  should  never  “nickel-dime"  the  program 
manager.  It  does  not  pay  in  the  long  run  to  be  over-restrictive  with  funds,  or  to 
require  excessive  justification  of  expenses.  Extra  dollars  put  into  a  training 
program  will  pay  off  in  various  places.  For  example,  in  this  study  it  was  clear 
that  the  757  program  was  not  only  fulfilling  its  primary  mission  by  turning  out  757 
pilots,  but  as  an  additional  benefit  was  producing  what  would  soon  become  the 
777  cadre.  This  was  an  unexpected  dividend  which  became  apparent  when  it 
was  time  to  put  the  777  program  in  place,  with  a  very  short  lead  time. 

Curriculum  Development  and  Standardization 

Determine  which  skills  will  be  taught  in  ground  school,  and  which  will  be 
considered  “hands  on”  and  left  to  IOE  and  line  checking.  It  is  very  easy  for 
ground  school  instructors  to  say,  “Don’t  worry  about  that  -  you’ll  get  it  on  the  line 
(or  in  IOE).”  This  attitude  can  result  in  serious  flaws  in  the  training  product,  and 
possibly  erroneous  learning  techniques  and  improper  procedures. 
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Likewise,  the  transitioning  pilot  tends  to  hear  on  the  line  at  many  companies,  “I 
don’t  care  what  they  taught  you  in  ground  school  and  simulator.  This  is  the  line 
out  here.  This  is  the  real  world.” 

There  must  only  be  one  standard,  and  it  must  be  taught  and  checked  constantly. 
It  is  a  failure  of  management  if  a  pilot  discovers  there  is  a  difference  between 
what  he  is  taught  in  ground  school  and  simulator  and  what  he  finds  on  the  line. 
All  instruction  that  is  given  a  pilot  must  be  line  oriented.  What  other  orientation 
could  possibly  be  entertained? 

We  have  often  advised,  standardize  the  airline,  not  the  pilots.  We  offer  the 
following  example.  At  one  large  carrier  (not  CAL)  where  we  did  some  work  on 
cockpit-cabin  communication,  the  flight  attendants  had  a  special  page  in  their 
manual  that  listed  the  four  pieces  of  information  that  the  cockpit  would  pass  to 
them  in  an  emergency  (e.g.  signal  to  brace,  time  to  prepare  The  first  letters  of 
the  four  formed  an  acronym.  The  page  was  considered  so  critical  that  it  was 
given  unique  color  and  a  nickname  based  on  that  color.  Unfortunately  the  pilots 
that  we  interviewed  had  never  heard  of  the  sheet  and  could  not  name  any  of  the 
four  items  the  cabin  needed. 

It  is  not  up  to  each  instructor  to  skip  over  a  lesson  or  avoid  questions  by 
reassuring  the  pilot  trainee  that  he  will  learn  this  during  his  IOE.  True,  there  are 
things  that  can  only  be  learned  on  the  line,  such  as  airport  environment,  taxi 
procedures,  and  communication  with  ground  crews.  But  what  the  pilot  is  taught 
and  where  and  when  it  is  taught  is  determined  by  the  training  syllabus,  not  the 
whim  and  judgment,  however  well  meaning,  of  each  instructor.  Again, 
standardize  the  airline,  not  the  pilots. 

No  single  program  can  optimally  serve  two  sub-populations,  one  with  and  one 
without  glass  experience.  We  believe  there  may  be  some  promise  in  developing 
a  semi-generic  introduction  to  glass  which  is  technical  instruction  material  on 
flight  deck  automation.  We  envision  a  pilot  taking  this  course  only  one  time, 
unless  there  were  changes  in  philosophy  at  the  company.  Taking  the  course 
only  once  would  eliminate  redundant  training,  which  is  both  costly  to  the 
company  and  frustrating  to  the  pilot.  The  answer  may  be  a  two-tiered  class  for 
transition  to  glass: 

1.  Past  Glass  (pilots  who  have  flown  some  glass  aircraft) 

2.  No  Glass  (pilots  who  have  never  flown  glass) 

This  would  allow  writing  a  syllabus  that  minimizes  the  re-teaching  of  materials 
already  learned  and  understood  by  the  first  group,  and  one  that  starts  at  “square 
one,”  teaching  basics  of  automation  and  glass  cockpits. 
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Documentation  and  Manuals 


An  airline  should  initially  draw  as  much  as  possible  from  the  manufacturers’ 
checklist,  procedures  manual,  MMEL,  and  other  documents.  But  ultimately  all 
procedures,  documents,  and  training  methods  must  be  tailored  to  the  operations 
and  philosophies  of  the  airline,  and  approved  by  the  FAA  principal  operations 
inspector  (POI).  Wherever  the  material  comes  from,  it  now  becomes  the 
standard  documentation  of  the  airline,  and  the  one  that  is  taught  and  reinforced. 
This  is  particularly  important  when  an  airline  goes  through  a  merger  or 
acquisition. 

We  again  recommend  that  cockpit  documentation  materials  be  subjected  to  a 
thorough  human  factors  study.  Specifically  we  recommend  that  NASA’s  human 
factors  experts  examine  the  QRH’s  and  checklists  at  various  air  carriers,  and 
make  recommendations  regarding  design  and  use  of  cockpit  documentation. 

An  industry  group  should  examine  the  products  of  the  "cottage  industries,”  with 
an  eye  toward  possibly  incorporating  them  into  the  airline  documentation,  with 
FAA  approval.  Flight  training  management  should  be  curious  as  to  why  these 
unofficial  and  uncertified  products  are  being  purchased,  when  the  official 
manuals  are  furnished  free  by  the  training  department.  One  company  told  us 
their  sales  of  FMS  manuals  are  in  the  thousands.  They  must  provide  something 
of  value,  in  the  mind  of  the  purchaser,  if  airline  pilots,  with  their  legendary 
reputation  for  penuriousness,  are  spending  their  own  money  on  these  products. 

Training  Devices 

[See  also  pp.  VIII-7  and  VIII-8  of  the  previous  chapter  for  a  discussion  of  faults 
and  recommended  improvements  in  training  devices  ] 

While  attending  ground  school  we  observed  that  the  CBT  devices  contained 
numerous  factual  errors  and  inconsistencies.  This  propagates  false  information, 
annoys  the  pilots,  and  diminishes  the  authority  of  the  CBT.  Every  effort  should 
be  made  to  detect  and  remove  these  errors  before  introducing  the  CBT,  by 
testing  on  a  cross-section  of  the  user  population,  not  just  ground  school 
instructors.  Management  should  be  scrupulous  about  minimizing  errors  in  all 
training  software.  We  also  noted  several  examples  where  the  CBT  programming 
was  not  supportive  of  the  student.  Some  of  these  have  since  been  corrected. 

A  qualified  ground  school  instructor  should  be  available  at  all  times  when 
students  are  studying  via  CBT.  (Failure  to  do  this  was  the  most  frequent 
complaint  about  CBT  training).  The  instructor  can  answer  questions,  and 
resolve  differences,  when  they  occur,  between  manuals  and  CBT  text. 
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Explore  the  potentials  for  web-based  training,  and  its  ability  to  centralize  the 
timing  and  distribution  of  new  or  revised  software.  Make  it  easy  for  pilots  to 
obtain  updated  software. 

Examine  the  use  of  FMC/CDU  part-task  simulators.  These  were  not  available  in 
this  study.  Especially  evaluate  “free  play"  (exploratory  learning)  on  these 
devices,  and  research  whether  free  play  or  a  more  structured  approach  is  the 
most  effective  utilization  of  part-task  simulators  Attempt  to  determine  the  cost 
effectiveness  of  the  FMC/CDU  devices,  compared  to  the  more  expensive  FTDs. 

We  have  found  in  our  interviews  and  questionnaires  a  high  incidence  of  FMC 
errors  in  the  first  year  on  the  line,  These  include,  but  are  not  are  not  restricted 
to,  failure  to  arm  LNAV  after  heading  selection,  confusion  over  the  various 
autopilot-autothrottle  modes,  confusion  over  VNAV  path  and  VNAV  speed,  often 
resulting  in  failure  to  make  a  crossing  restriction,  and  the  need  to  update  winds 
We  feel  that  a  part-task  simulator  with  opportunities  for  “free  play"  should  be 
examined  as  a  potential  remedy  for  these  errors. 

CRM  Training 


Attempt  to  shield  CRM  from  “company  politics."  CRM  training  (and  trainers) 
seem  to  be  particularly  vulnerable  to  the  changing  whims  of  management  at  the 
carriers  we  have  visited.  CRM  instructional  programs  rise  and  fall  very  quickly. 
This  may  be  due  in  part  to  the  fact  that  these  programs  are  not  well  understood 
by  airline  management  or  the  FAA.  It  may  also  be  due  to  the  fact  that  the  goals 
are  vague,  and  the  methods  appear  to  be  rooted  in  psychotherapy,  in  an 
industry  that  values  engineering  over  social  sciences.  The  instability  of  CRM 
instruction  may  also  be  attributed  to  the  difficulty  in  finding  “hard”  measures  of 
success  and  of  value  versus  cost. 

It  is  not  easy  to  say  how  CRM  programs  can  be  protected  from  political  whim, 
but  we  have  little  doubt  that  it  must  be  done  or  the  effectiveness  of  CRM 
instruction  will  suffer  and  the  costs  will  increase.  [Imagine  that  at  an  airline, 
once  every  few  years,  there  occurred  a  movement  to  radically  change  the 
teaching  of  hydraulics,  and  the  proponents  of  the  present  methods,  along  with 
their  teaching  programs,  syllabi,  and  materials  were  swept  away  to  make  room 
for  the  new!] 

Ab  Initio  and  Very  Low-Time  Trainees 

If  ab  initio  training  results  in  the  hiring  of  low-time  pilots  for  sophisticated 
cockpits,  the  airline  must  carefully  examine  its  training  program.  The  training 
appropriate  for  the  more  experienced  pilot  new-hire  may  be  inappropriate  for  an 
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ab  initio  graduate.  It  would  be  wise  for  any  airline,  contemplating  hiring  ab  initio 
pilots  to  become  familiar  with  the  experience  of  the  European  and  Asian  airlines 
which  have  years  of  experience  with  low-time,  new-hire  pilots. 

Style  of  Instruction 

There  is  disagreement  in  the  industry,  and  among  the  three  authors  of  this 
report,  on  where  to  put  CRM  and  H-CAT  in  the  syllabus.  One  side  says  that  If 
there  is  going  to  be  an  instructional  block  on  CRM,  human  factors,  or  philosophy 
of  automation,  it  should  be  offered  in  the  first  session  of  the  first  day  in  the 
ground  school  program  for  three  reasons:  (1)  this  is  the  most  effective  place  for 
introductory  material;  (2)  it  prepares  the  student  to  "think  CRM"  from  the  very 
first  session  on  the  FTD  and  incorporate  CRM  concepts  into  his/her  behavior; 
and  (3)  once  the  student  is  exposed  to  aircraft  systems,  it  is  too  late.  The  typical 
pilot  then  does  not  want  to  be  exposed  to  anything  else.  The  pressure  to  pass 
the  oral  exam  starts  to  build. 

Another  equally  respectable  view  prefers  to  place  the  systems  instruction  first, 
let  the  pilots  take  their  oral  exam  and  put  that  behind  them,  and  then  turn  to  the 
CRM/H-CAT  block,  which  the  student  can  now  learn,  freed  of  “orals  stress." 

Develop  a  culture  of  helpfulness  on  the  part  of  instructors  toward  the 
transitioning  students.  Discourage  any  non-constructive  behavior  on  the  part  of 
the  instructors.  The  “helpfulness"  and  “friendliness”  of  the  CAL  757 
instructional  personnel  and  the  Boeing  instructors  was  commented  frequently  by 
the  pilots  in  this  study,  both  in  questionnaires  and  in  interviews.  Alterations  in  a 
company  culture  do  not  come  easily  and  cheaply. 

Administration  and  Scheduling 

Distribution  of  pre-ground  school  information  may  help  reduce  apprehension  and 
misinformation.  Materials  could  be  mailed  to  each  pilot  scheduled  to  attend 
transition  training  for  glass  for  the  first  time.  Pilots  frequently  ask  for  manuals  for 
the  new  plane  prior  to  transition  for  the  plane.  This  request  should  be 
accommodated.  Perhaps  the  answer  is  to  provide  each  pilot  on  the  roster  for  a 
future  transition  to  glass  with  a  “pre-training  package”,  of  what  to  expect,  a 
syllabus,  reading  materials,  and  a  schedule  of  events.  Some  of  our  pilot  group 
called  this  a  “heads-up  package.” 

Pilots  should  also  be  encouraged  to  take  a  jumpseat  ride  in  the  model  they  will 
be  flying,  or  if  not  possible,  in  any  glass  aircraft,  prior  to  transition  training.  If  this 
is  not  deemed  practical,  it  may  be  possible  to  schedule  the  trainee  to  observe  a 
simulator  session. 
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At  some  carriers,  pilots’  unions  have  questioned  recommendations  such  as  the 
above  on  the  grounds  that  they  constitute  work  without  pay.  This  is  may  be  a 
valid  point  which  must  be  worked  out  by  management  and  labor.  We  have  no 
recommendations  on  this  matter,  as  labor-management  contractual  affairs  are 
outside  the  scope  of  this  study. 

Minimize  delays  between  transition  training  and  IOE  or  line  assignment.  Also 
attempt  to  minimize  time  between  transition  training  and  the  rating  ride. 

Avoid  if  possible  sending  newly  transitioned  pilots  back  to  their  old  planes. 

Scheduling  should  provide  mixed  lines  of  domestic  and  overseas  flying  for  all 
fleets  that  do  both.  This  will  keep  proficiency  for  both  types  of  trip  at  a  higher 
level  than  flying  “pure”  lines.  This  is  particularly  important  for  those  who  tend  to 
bid  only  trans-oceanic  ETOPS  flying,  and  whose  basic  flying  skill  (and  possibly 
automation  skills)  may  suffer  from  flying  only  very  long  legs.  The  use  of 
international  relief  pilots  makes  this  all  the  more  critical.  A  trip  with  a  few  long 
legs  and  an  augmented  crew  to  share  the  approaches  and  landings  also  raises 
the  issue  of  proficiency  loss.  Mixing  domestic  and  international  lines  in  trips  will 
relieve  this  problem,  and  probably  at  little  cost. 

Checking  and  Evaluation 

Explore  the  potential  of  using  Flight  Operations  Quality  Assurance  (FOQA)  data 
as  a  source  of  feedback  for  the  effectiveness  of  the  training  program.  The  flight 
crews  must  be  in  no  jeopardy  from  company  or  FAA  enforcement  action.  The 
first  time  FOQA  data  are  used  against  a  pilot,  the  program  will  come  to  a  stop. 

Explore  the  use  of  electronically  based  FAQs  (Frequently  Asked  Questions)  for 
ongoing  design  and  development  of  training  programs. 

Do  not  restrict  check  procedures  to  full-up  automation.  Doing  so  can  have 
several  negative  results,  the  most  critical  being  that  it  deprives  the  crew  of 
revealing,  and  the  check  airman  from  observing,  the  most  important  aspect  of 
automated  flying,  the  crew’s  choice  of  modes  and  options.  Crews  should  fly  a 
checkride  as  they  would  any  other  leg  on  a  trip,  and  in  accordance  with  company 
policies  on  use  of  automation,  utilizing  the  autopilot/autothrottle  mode  that  they 
consider  appropriate  to  the  circumstances.  If  they  are  not  allowed  to  do  this, 
and  are  required  to  fly  “full-up”,  not  only  are  they  forced  to  fly  in  an  artificial  and 
perhaps  unsafe  configuration,  but  also  the  check  airman  is  deprived  of  the  most 
valuable  data  on  which  to  evaluate  the  crew. 
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The  training  programs  themselves  should  be  subjected  to  continuous  evaluation. 

Encourage  feedback  from  the  student  pilots  and  evaluation  of  all  training 
programs,  from  ground  school  and  simulator  through  IOE  and  rating  rides, 
including  recurrent  training.  We  recommend  a  formal,  structured  process  for  a 
running  evaluation  of  all  training  programs.  The  evaluation  should  probably  be 
done  by  an  outside  agent,  reporting  high-up  in  the  training  and  standardization 
hierarchy.  The  results  of  these  evaluations  must  not  be  mere  “number 
crunching”  for  its  own  sake,  but  should  affect  changes  in  the  training  program 
(e.g.  curriculum,  use  of  devices,  instructional  methods).  Thus  the  training 
department  creates  an  instructional  feedback  loop  that  should  result  in 
continuous  improvement  and  quality  management  in  the  training  program  and 
product. 

IOE 

Many  pilots  reported  that  the  first  IOE  should  not  be  an  Atlantic  crossing,  as 
there  was  too  much  to  learn,  bordering  on  overload.  They  felt  that  the  first  IOE 
leg  after  transition  training  should  be  a  normal,  domestic  leg.  Introduce 
international  flights  and  especially  ETOPS  flights  only  after  the  "normal” 
domestic  IOE  has  been  completed. 

Our  analysis  of  the  NASA-ASRS  reports  (sampled  from  their  entire  database, 
names  of  the  carriers  unknown)  isolated  IOE  as  an  instructional  phase  prone  to 
problems,  namely  competing  priorities  that  could  jeopardize  the  safety  of  the 
flight.  The  IOE  check  airman  has  two  possibly  conflicting  duties:  safely  flying  a 
revenue  flight,  and  rendering  instruction  to  the  newly  transitioned  trainee.  Pilots 
in  this  study  regarded  IOE  as  an  important  learning  experience.  However,  the 
role  of  the  check  airman  as  an  instructor  must  not  be  allowed  to  vie  with  his 
simultaneous  responsibility  to  fly  the  aircraft  in  the  safest  manner. 

We  do  not  know  the  answer  to  this  complicated  question,  and  have  no 
recommendation  except  that  the  problem  be  studied  by  both  operations  experts 
and  human  factors  experts.  Obviously  the  newly  transitioned  pilot  has  to  start 
somewhere,  but  perhaps  the  present  IOE  structure  is  not  the  safest  way  to  go, 
and  may  not  be  most  conducive  to  integrating  skills  and  knowledge  recently 
learned.  Further  research  should  be  conducted  to  identify  the  specific  phases  of 
flight  and  procedures  that  are  most  vulnerable  to  conflicts,  and  how  these 
conflicts  might  best  be  handled.  Additional  research  should  contribute  to  the 
design  of  strategies  for  the  enhancement  of  line  instruction  while  ensuring  the 
safety  of  flight. 
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Early  Line  Experience 


Some  routes  are  more  difficult  than  others  (e.g.  more  sidesteps,  last-minute 
changes  in  the  terminal  area,  holding,  vectors,  noise  abatement  arrivals  and 
departures,  and  more  difficult  airports).  Scheduling  should  assign  easier,  less 
demanding  routes  to  inexperienced  crew  members  (somewhat  in  the  same 
manner  as  less  demanding  airports  for  high-minimums  captains).  We  recognize 
that  this  recommendation  places  a  heavier  burden  on  the  scheduling 
department,  but  we  feel  that  the  benefits  to  training  and  safety  would  be  worth 
the  cost. 

Consider  scheduling  a  one-day  follow-up  training  session  at  some  time  during 
each  pilots’  first  year  on  the  line.  The  session  would  be  used  to  answer 
questions,  expand  on  automation  techniques,  and  reinforce  little-used  or  weak 
procedures  and  skills.  A  program  of  this  sort  would  be  very  expensive,  and  we 
recommend  only  that  its  costs  and  benefits  be  considered. 

Cockpit  Environment 

Engender  the  concept  of  “mutual  aid”  in  line  pilots,  as  contrasted  with  “separate 
work  stations”  with  only  the  procedurally  required  communication.  This  is 
especially  important  when  one  or  both  pilots  are  inexperienced  (e.g.  recently 
post-IOE). 

Pilots  should  not  hesitate  to  reveal  to  each  other  on  initial  contact  their 
experience  level  on  the  new  aircraft.  We  recommend  that  this  be  addressed  as 
a  form  of  CRM  training.  At  times  great  differences  in  time-in-type  will  be 
encountered  on  the  flight  line.  This  should  be  known  by  both  pilots,  as  one  may 
have  to  compensate  for  the  inexperience  of  the  other.  It  is  probably  an 
endorsement  of  the  CRM  training  that  captains  we  have  observed  have  little 
hesitation  in  saying  to  the  first  officer,  in  so  many  words,  “I’m  new  at  this  (glass 
cockpit)  and  I’d  appreciate  all  the  help  you  can  give  me.”  Such  a  statement  we 
would  regard  as  good  planning,  good  briefing,  and  good  CRM,  and  it  should 
promote  a  more  relaxed  cockpit  atmosphere. 

Briefings 

The  importance  of  briefings  should  be  taught  and  emphasized  all  along  the  way 
in  transition  training.  They  are  the  foundation  of  effective  communication  and 
the  proper  performance  of  duties  in  the  cockpit.  Briefings  should  be 
demonstrated  by  FTD  and  FSS  instructors,  check  airmen  and  IOE  instructors, 
and  the  students  should  practice  briefing  in  every  session.  The  connection 
between  briefings  and  CRM  should  be  noted.  Stress  the  importance  of  briefing 
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by  the  captain  (or  PF)  to  the  other  pilot(s)  and  to  the  lead  flight  attendant. 
Training  for  briefings  should  be  part  of  any  AQP  proposal. 

Flight  attendant  briefings  should  also  be  taught  and  practiced.  Topics  should 
include,  but  not  be  limited  to;  (1)  management  of  unruly  passengers;  (2) 
suspension  of  service  during  turbulence;  (3)  clarification  of  sterile  cockpit 
procedures;  (4)  anticipated  weather;  (5)  communication  between  the  cockpit  and 
the  cabin.  Captain/flight  attendant  briefings  are  a  two-way  street:  the  lead  F/A 
should  brief  the  captain  on  any  special  problems  or  requirements  which  he/she 
anticipates  for  the  flight  (e.g.  heavy  passenger  load,  meal  service  on  a  short 
leg.)  For  a  good  example  of  a  captain’s  F/A  briefing,  see  Chute  and  Wiener 
(1996,  p.  226). 

Briefing  of  flight  attendants  should  be  an  integral  part  of  the  pilot  training 
syllabus  and  the  FOM,  and  practiced  during  transition  training,  including  during 
FTD  exercises.  Annual  recurrent  training  may  be  a  good  time  to  review  the 
contents  and  technique  of  briefings. 

Planning  is  closely  related  to  briefing.  It  is  particularly  important  in  two-pilot, 
glass  aircraft,  and  is  the  foundation  of  workload  management.  Planning  should 
be  emphasized  in  the  H-CAT  program,  and  in  FTD  and  FSS  training.  The  FTD 
is  the  place  to  learn  and  test  planning  techniques 

Method  of  Flying  and  Flight  Safety 

Flight  management  should  formulate  a  policy  on  maintaining  manual  (hand 
flying)  skills,  and  convey  this  to  the  pilots.  Hand  flying  of  the  new  aircraft  during 
transition  training  is  highly  desirable,  and  some  portion  of  the  simulator  training 
should  be  devoted  to  this.  Guidelines  for  hand  flying  should  be  developed, 
specifying  where  it  can  be  done,  under  what  weather  conditions,  and  what  types 
of  approaches.  As  always,  the  captain’s  discretion  prevails.  We  must  not  forget 
that  hand  flying  can  have  negative  consequences  as  well  (e.g.  high  workload 
and  failure  to  scan  the  “big  picture.”) 

Allow  for  the  practice  of  non-automation-based  problem-solving  skills  and 
infrequently  used  procedures. 

In  all  flights,  observe  fundamental  rules  of  safe  piloting.  Several  interviewees 
commented  that  pilots  were  not  clearing  turns  in  VMC.  This  precaution  goes 
back  to  the  first  day  any  pilot  began  his  flight  training.  One  pilot  remarked,  “At 
least  glance  in  the  direction  you  are  turning.” 
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Consider  the  FMC/MCP  as  a  control  to  be  "handed  off  to  the  other  pilot,  and 
formalize  the  handoff,  perhaps  with  a  callout  such  as  “your  FMC,”  to  provide 
feedback  that  a  transfer  has  been  completed. 

Workload  Management 

Management  should  clarify  the  policy  and  procedure  on  PF/PM  duties  and 
sharing  of  work  load.  It  is  critically  important  in  the  glass  cockpit  to  specify 
clearly  “who  does  what”  and  to  conform  to  procedures,  and  to  stick  to  the  task 
assignments  [At  Continental,  the  term  “pilot  monitoring”  (PM)  has  replaced  the 
familiar  “pilot  not  flying”  (PNF)]. 

We  recommend  that  airlines  consider  replacing  the  term  “pilot  not  flying”  (PNF) 
with  “pilot  monitoring,”  as  CAL  has  done.  It  gives  the  position  a  more  positive 
duty,  stressing  what  the  pilot  does,  rather  than  what  he/she  does  not  do.  It 
further  enhances  the  task  of  monitoring,  an  increasingly  important  activity  in  the 
age  of  automation. 

Reduce,  through  systems  analysis,  the  frequency,  complexity,  and  length  of 
“company  reports”,  especially  at  low  altitudes  or  in  crowded  airspace,  whether 
they  are  done  by  voice  transmission  or  by  data  link.  Routine  calls  could  be 
made  at  high  altitude,  or  preferably  at  cruise.  Many  calls  can  be  made  via 
existing  datalinks  (e.g.  ACARS),  but  we  are  not  prepared  to  say  that  this  is  a  big 
improvement  over  traditional  voice  transmissions.  Use  of  ACARS  probably  does 
solve  the  problem  of  frequency  saturation  that  occurs  in  terminal  areas,  but 
preparation  and  transmission  of  an  ACARS  message  is  time  consuming.  In  the 
near  future  much  more  sophisticated  datalink  systems  will  be  available,  and 
standard  messages  can  be  stored  and  sent,  with  a  minimum  of  keystrokes, 
possibly  reducing  cockpit  workload. 

Flight  management  should  examine  all  company  business  required  of  the  cockpit 
crew,  with  an  eye  toward  minimizing,  or  better  yet,  eliminating,  duties  that  must 
be  performed  below  18,000  feet.  This  recommendation,  in  one  form  or  another, 
appeared  in  various  places  in  this  study. 

Consider  placing  some  type  of  terminal,  either  voice  or  datalink  (e.g.  ACARS)  in 
the  cabin  for  flight  attendants  to  use  for  passenger  matters.  We  see  no  reason, 
other  than  tradition,  why  F/As  have  to  come  to  the  cockpit  with  requests  for  gate 
information,  galley  supplies,  or  wheel  chairs. 
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Impact  on  Future  Air  Traffic  Management 


We  have  noted  throughout  this  study  that  pilots  are  having  difficulty  with 
navigation,  specifically  VNAV,  in  achieving  level-offs,  meeting  crossing 
restrictions,  and  initiating  descents.  The  FAA  should  take  note  of  these 
implications  for  the  design  of  future  air  traffic  management  systems  such  as 
“free  flight”. 

Hardware  Standardization 


Where  possible,  standardize  over  models  by  the  same  manufacturer.  This  will 
cut  cost  and  time  required  in  training  pilots,  cabin  crews,  and  maintenance 

personnel.  It  may  also  enhance  the  quality  of  maintenance,  and  make  easier  the 
cross-qualification  of  crews.  It  may  also  allow  cross-utilization  of  simulators  to 
train  for  various  models. 
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EPILOGUE 


In  this  study  we  have  examined  the  development  and  installation  of  a  training 
program  designed  to  transition  pilots  from  old  technology  to  new,  computer- 
based  cockpit  technology.  We  have  commented  on  what  is  good  and  what  is 
not.  We  feel  that  the  success  of  the  program,  its  nearly  zero  failure  rate,  and  its 
acceptance  by  the  pilots  is  the  result  of  the  right  people,  the  meticulous  design 
of  the  program,  and  the  strong  support  of  management.  The  “clean  sheet  of 
paper”  policy  has  obviously  paid  off. 

Pilot  opinion  is  not  everything,  but  measured  correctly,  it  is  one  valuable 
assessment  of  the  program.  Shown  below  is  one  of  the  graphic  displays  of  pilot 
opinion,  taken  after  slightly  over  a  year  on  the  line.  The  results  speak  for 
themselves. 


22c.  Training  for  the  757  was  as  adequate 
as  any  training  that  I  have  had. 


so 


We  add  only  that  in  prior  field  studies  involving  various  airlines  in  the  U.S.,  with 
over  40  opinion  probes  administered  to  hundreds  of  pilots,  we  have  never  seen 
a  response  this  extreme  on  any  subject,  nor  such  an  endorsement  of  a 
company’s  pilot  training  program.  The  authors  give  credit  to  the  men  and 
women  throughout  Continental  Airlines  for  this  success.  We  hope  that  others 
will  be  guided  by  Continental’s  experience,  and  will  not  only  make  use  of  these 
results,  but  will  make  improvements.  Aviation  safety  comes  not  in  dramatic 
breakthroughs,  but  in  slow,  cautious,  sometimes  tedious,  step-by-step  expansion 
of  what  we  already  know. 
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8.  We  also  recognize  the  support  of  the  staff  of  NASA’s  Aviation  Safety  Reporting 
System.  The  reports  quoted  in  the  chapters  and  in  Appendix  H  came  from  a 
search  that  we  requested.  We  sought  reports  in  which  the  incident  contained 
elements  of  two  factors:  automation  and  training.  We  again  wish  to  make  it  clear 
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APPENDIX  A 


Likert  Attitude  Scales  for  Three  Phases  of  the  Study 


On  each  of  the  following  pages  in  this  appendix,  the  data  from  the  Likert  attitude  scales 
are  displayed  in  graphic  form.  There  is  a  plot  for  each  probe,  and  each  phase  of  the 
study.  The  page  numbers  correspond  to  the  probe  numbers  (e.g.  Probe  no.  13  “There 
is  too  much  workload  below  10,000  feet  and  in  the  terminal  area”  can  be  found  on  page 
A-13). 

For  the  first  20  pages  (A-1  to  A-20),  there  are  three  plots  per  page,  in  the  following 
order: 


PHASE  1  (first  day  of  ground  school) 
PHASE  2  (about  3-4  months  after  training) 
PHASE  3  (about  12-14  months  later) 


Thus  the  reader  can  compare  the  graphs  over  time.  For  a  graphic  presentation  of  the 
phases  and  sample  sizes,  see  Figure  Ill-Ion  Page  111-4. 

Probes  21  through  24  were  administered  only  in  Phase  2  and  3,  as  the  probe  was 
inappropriate  for  pilots  not  yet  out  on  the  time  in  the  757.  These  pages  (A-22  through 
A-24  display  only  those  two  graphs. 
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A-1 


2b.  I  am  concerned  that  automaton  wit 
cause  me  to  lose  my  flying  stals. 


A-2 


A-3 


4a.  R  ts  Important  to  me  to  fly  the  most  modem 
plane  In  my  company's  fleet 

40  - - - — - - 


4b.  I  Is  important  to  fly  the  most  modem 
plane  in  my  company's  fleet 

40  t  -  "■  11  -  ■■  ■  ■  - . -  1 


4c.  I  is  important  to  fly  the  most  modem 
plane  in  my  company's  fleet. 

40  - — 


A-4 


5a.  As  I  took  at  aircraft  today,  Ithink  the/ve 
gone  too  far  with  automation. 

ao - - - 


5b.  As  I  took  at  aircraft  today,  Ithink  theyVe 
gone  too  far  wth  automation. 


5c.  As  I  took  at  aircraft  today,  Ithink  theyVe 

gone  too  far  wth  automation. 

- — - - - - , - - 
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6a.  -Automated  cockpits  require  more  cross-checking 
of  crew  members'  actions. 
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40 
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6c.  Automated  cockpits  require  more  cross-checking 
of  crew  members'  actions. 
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7a.  In  the  aircraft  I  am  presently  flying,  it  is  easy  for 

the  captain  to  monitor  and  supervrse  the  first  officer. 
•0 - - - - - - - 


7b.  h  the  aircraft  I  am  presently  flying,  ft  is  easy  for 
the  captain  to  monitor  and  supervise  the  first  officer. 

%  i - - - - - - - 


7c.  h  the  aircraft  I  am  presently  flying,  ft  is  easy  for 
the  captain  to  monftor  and  supervise  the  first  officer. 

B* - 
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14a.  Automaton  does  not  reduce  total  workicad 
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9a.  Automation  frees  me  of  much  of  the  routine, 
mechanical  parts  of  flying  so  I  can 

concentrate  on  “managing"  the  flight. 

to - 


9b.  Automation  frees  me  of  much  of  the  routine, 
mechanical  parts  of  flying  so  l  can 
concentrate  on  "managing"  the  ffight 


9c.  Automation  frees  me  of  much  of  the  routine, 
mechanical  parts  of  flying  so  I  can 
concentrate  on  "managing"  the  ffight. 
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10a.  I  am  not  concerned  about  making  errors,  as  long 

as  we  follow  procedures  and  checkfsts 
s - 


10b.  I  am  not  concerned  about  making  errors,  as  long 
as  we  follow  procedures  and  checklists. 

40  « - - - - - - 


10c.  I  am  not  concerned  about  making  errors,  as  long 
as  we  follow  procedures  and  checklists. 
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1  la.  I  k>ok  forward  to  more  automation 
—  the  more  the  better. 


1 1  b.  I  took  forward  to  automation 
—  the  more  the  better. 


1 1  c.  I  took  forward  to  automation 
—  the  more  the  better. 

40  - - 


A-11 


12a.  I  have  no  trouble  staying  "ahead  of  the  plane". 


12c.  I  have  no  trouble  staying  "ahead  of  the  plane" 
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13a.  There  ts  too  much  workload  below 
10,000  feet  and  in  the  terminal  area. 


13b.  There  is  too  much  workload  below 


10,000  feet  and  in  the  terminal  area. 


40 
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A-14 


A-15 


1 6a.  I  am  concerned  about  the  reltabirty  of 

some  of  the  automation  equipment. 

- - 


16b.  I  am  concerned  about  the  reliabiity  of 
some  of  the  automation  equipment 

■i - - 
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16c.  I  am  concerned  about  the  reiabilly  of 
some  of  the  automaton  equipment 

50  - 
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17a.  I  am  concerned  about  the  lack  of  tone  to 
look  outside  the  cockpit  for  ether  aircraft. 


17b.  lam  concerned  about  the  lack  of  time  to 

look  outside  the  cockpit  for  other  aircraft. 

80  - 


17c.  lam  concerned  about  the  lack  of  time  to 
look  outside  the  cockpit  for  other  aircraft. 
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18b.  ContinentaPs  CRM  training  has  been  helpful  to  me. 
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18c.  ContinentaPs  CRM  training  has  been  helpful  to  me. 
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19a.  Some  times  I  feel  more  like  a 
"button  pusher*  than  a  pitot. 


19b.  Sometimes  I  feel  more  Ike  a 
"button  pusher"  than  a  pilot. 
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20a.  I  regularly  maintain  flying  proficiency  by 
turning  off  automaton  and  hand  flying. 


20b.  I  regularly  maintain  flying  proficiency  by 
turning  off  automation  and  hand  flying. 


20c.  I  regularly  maintain  flying  proficiency  by 
turning  off  automation  and  hand  flying. 

eo  «< - - 
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A-21 


22b,  Training  for  the  757  was  as  adequate 
as  any  training  that  t  have  had. 

90  — - “ 


22c.  Training  for  the  757  was  as  adequate 
as  any  training  that  I  have  had. 

»  - - - - - - 
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23b.  Electronic  flight  instilments  f  glass  cockpits") 
are  a  big  advance  for  flight  safety 


23c.  Electronic  flight  instum ents  f  glass  cockpits") 
are  a  big  advance  for  flight  safety. 

80  — - - 
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24b.  There  are  still  modes  and  features  of  the  757 

automation  that  I  do  not  understand. 

* - - - 


24c.  There  are  still  modes  and  features  of  the  757 
automation  that  I  do  not  understand. 
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APPENDIX  B 


AC 

ACARS 

ADF 

ADI 

AFCS 

AFDS 

Al 

AOA 

AQP 

ARINC 

ASRS 

CBT 

CDU 

CFIT 

CRM 

CRT 

CVR 

ECAM 

EEC 

EFIS 

EGPWS 

El  CAS 

ELS 

ELT 

ETOPS 

FAR 

FBS 

FFS 

FMA 

FMC 

FMEA 

FMS 

FOQA 

FTD 


Glossary  of  Abbreviations 


Advisory  circular  (FAA) 

ARINC  communication  and  reporting  system 

automatic  direction  finder 

attitude  director  indicator 

automatic  flight  control  system 

automatic  flight  director  system 

artificial  intelligence 

angle  of  attack 

advanced  qualification  program 
Aeronautical  Radio,  Inc. 

Aviation  Safety  Reporting  System  (NASA) 
computer-based  training 
control-display  unit 
control  flight  into  terrain  (accident) 

cockpit  resource  management;  crew  resource  management 
cathode  ray  tube 
cockpit  voice  recorder 

electronic  centralized  aircraft  monitor  (Airbus) 
electronic  engine  control 
electronic  flight  instrument  systems 
enhanced  GPWS 

engine  indication  and  crew  alerting  system  (Boeing) 

electronic  library  system 

emergency  locator  transmitter 

extended  two-engine  operations 

federal  aviation  regulation 

fixed  base  simulator 

full  flight  simulator 

flight  mode  annunciator 

flight  management  computer 

failure  mode  and  effects  analysis 

flight  management  system 

flight  operations  quality  assurance 

flight  training  device 
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GPS 

global  positioning  system 

GPWS 

ground  proximity  warning  system  (see  also  EGPWS) 

H-CAT 

human-centered  automation  training  (Continental) 

HSI 

horizontal  situation  indicator 

INS 

inertial  navigation  system 

IRO 

international  relief  officer 

IRS 

inertial  reference  system 

IRU 

inertial  reference  unit 

LNAV 

lateral  navigation 

LOFT 

line  oriented  flight  training 

LOS 

line  oriented  simulation 

MEL 

minimum  equipment  list 

MMEL 

master  MEL 

MCP 

mode  control  panel 

MOA 

military  operations  area 

MS  AW 

minimum  safe  altitude  warning 

NATRAC 

North  Atlantic  tracks 

NDB 

non-directional  beacon 

PC 

proficiency  check 

PF 

pilot  flying 

PIP 

product  improvement  package  (Boeing) 

PM 

pilot  monitoring  (see  also  PNF) 

PNF 

pilot  not  flying 

POI 

principal  operations  inspector  (FAA) 

QRH 

quick  reference  handbook 

RNAV 

area  navigation 

RT 

recurrent  training 

TCAS 

traffic  alert/collision  avoidance  system 

TMC 

thrust  management  computer 

VNAV 

vertical  navigation 

WPT 

waypoint 
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Marginal  Homogeneity  Tests:  Q1  vs.  Q2 


The  eight  pages  that  follow  show  graphically  the  results  of  the  marginal  homogeneity 
tests,  and  histograms  of  the  before  and  after  Likert  scales.  Only  the  eight  probes 
resulting  in  significant  differences  on  the  marginal  homogeneity  test  are  included.  The 
table  on  the  bottom  half  of  the  page  displays  the  homogeneity  matrix. 

As  explained  previously,  if  there  is  no  change  in  attitude,  the  tally  would  be  in  the  main 
diagonal.  Taking  for  example  page  C-1,  there  were  15  pilots  who  strongly  agreed  with 
the  probe  in  Phase  1 ,  and  again  in  Phase  2.  There  were  9  pilots  who  changed  from 
“strongly  agree”  to  just  “agree”.  Off-diagonal  tallies  indicate  changes  in  attitude.  The 
further  the  tally  from  the  main  diagonal,  the  greater  the  pilot’s  change  in  attitude  from 
Phase  1  to  Phase  2.  For  example,  on  page  C-1,  in  response  to  the  probe  (“Flying 
today  is  more  challenging  than  ever”)  four  pilots  changed  their  vote  from  “strongly 
agree”  t  o  “disagree,”  a  rather  extreme  change. 

The  top  graph  is  similar  to  the  other  Likert  plots  that  have  been  shown,  except  that  two 
sets  of  data  are  included  in  each  graph:  Phase  1  and  Phase  2.  These  plots  give  the 
reader  a  comparison  of  mean  responses  from  the  two  phases. 
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Phase 


7.  In  the  aircraft  I  am  presently  flying,  it  is  easy  for 
the  captain  to  monitor  and  supervise  the  first  officer. 
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APPENDIX  D 


Questionnaire  Forms  -  Q1,  Q2,  and  Q3 


The  following  pages  contain  the  three  questionnaire  forms  (known  as  Q1,  Q2,  and  Q3), 
from  the  three  phases  of  the  study  (PI ,  P2,  and  P3).  For  brevity  the  three  are  not 
shown  in  their  entirety,  since  there  is  considerable  repetition  of  attitude  probes  lists  and 
answer  forms,  as  well  as  repetition  in  Q2  and  Q3  of  open-ended  questions. 
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NASA/CONTINENTAL  QUESTIONNAIRE  NO.  1 


Date  you  started  757  transition:  _  _  _ 

MM  DD  YR 

Name _ Capt.,  F/O,  S/O 

Home  Address _ 

City  _ 

State  _  ZIP  _ 

Present  Base _  Base  after  757  transition _ 

Home  Phone:  Area  Code  and  number:  (  )  _ 


Make  up  an  ID  code  for  yourself  and  enter  it  below.  Use  any  combination  of  letters  and 
numbers  (up  to  a  max  of  6).  Do  not  use  your  Social  Security  or  company  pay  number, 
birth  date,  etc.  Insert  it  in  the  blank  below.  The  characters  in  the  last  two  positions  are 
reserved  for  our  purposes.  The  full  eight  characters  make  up  your  ID  (e.g. 

ELW8681 5).  If  you  use  less  than  six  characters,  still  include  the  last  two  (e.g. 
TOM415). 

The  red  sticker  is  for  you  to  keep  a  record  of  your  ID.  Please  enter  your  full  ID  as 
written  below  onto  the  red  tag  and  keep  it  some  convenient  place.  We  suggest  a  log 
book  or  Jep  manual.  If  you  have  questions,  please  call  the  project  director  or  your 
IACP  Safety  Committee.  Remove  and  keep  the  red  tag.  It's  yours! 


Informed  consent: 

I  have  read  and  understood  the  material  in  the  attached  booklet,  including  the  purpose 
and  method  of  the  study,  and  I  consent  to  serve  as  a  volunteer  pilot. 

Signed:  _ 


ID  Code: 


1  5 
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I.  BIOGRAPHIC  DATA  AND  AIRCRAFT  EXPERIENCE 


This  is  our  first  effort  to  collect  some  information  from  you  about  your  attitudes  toward 
cockpit  technology  and  your  experiences.  First  we  need  some  information  about  you, 
and  about  your  flight  experience. 

1 .  Your  present  age  to  closest  month:  _ years _ months 

2.  Gender  (circle  one):  M  F 

3.  We  would  like  to  know  your  past  experience  in  Continental  turbojet  aircraft. 
Please  consider  your  experience  only  at  Continental  Place  an  "X"  in  the  box  for 
each  seat  on  each  aircraft  that  you  have  ever  flown  at  Continental.  Do  not  put 
flying  time. 


AIRCRAFT  SEAT 


Captain 

F/O 

S/O 

DC-9 

— 

MD-80 

— 

B737-1 00/200 

B737-300 

B727 

A-300 

B707/720 

DC-10 

B747 

4.  Which  seat  in  which  aircraft  did  you  occupy  immediately  before  going  to 
757  transition.  If  it  is  a  B-737,  please  state  model.  How  many  months 
had  you  been  in  this  seat? 

Aircraft _  Seat _  Months _ 

5.  Approximate  total  flying  hours  at  Continental  (include  S/O) 

_ hours 
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6.  Approximate  total  flying  hours,  all  aircraft  (airline,  military,  general  aviation  etc.) 

_ hours 

7.  When  did  you  attend,  or  when  do  you  plan  to  attend  B-757  transition  training? 

Month  _  Year _ 

8.  List  in  the  space  below  any  glass  cockpit  (EFIS)  aircraft  that  you  have  flown 
(airline,  commuter,  military,  corporate) 


9.  What  do  you  consider  the  most  advanced  aircraft  (with  respect  to 

instrumentation,  avionics,  automation  etc.)  that  you  have  flown?  Include  military 
or  other  employers: 

Answer: _ 


10.  Do  you  use  a  personal  computer  at  home?  (Y/N)  Ans: _ 

11.  If  yes,  when  you  are  at  home,  how  often  do  you  use  it?  (check  one) 

Daily _  Several  times  a  week _  Less  than  weekly _ 

12.  Approximately  how  many  actual  Cat  II  approaches  did  you  make  (as  PF  or  PNF) 
last  calendar  year?  Autolands? 

Cat  II  _  Autolands _ 

13.  Approximately  how  many  non-precision  approaches  (as  PF  or  PNF)  did  you 
make  last  calendar  year? 

VOR  _  LOC  _  ADF  _ 

14.  If  the  money  and  quality  of  trips  were  all  the  same,  and  base  assignment  were 
not  a  factor,  what  would  be  your  first  choice  of  plane  to  fly  in  Continental's  fleet? 
(Include  B-737-500  and  B-757  as  present  fleet.) 

Aircraft:  _ 
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II.  ATTITUDE-TOWARD-AUTOMATION  SCALE 


This  is  a  20-item  attitude  scale.  It  is  called  an  "intensity  scale"  because  you  can 
indicate  not  only  your  agreement  or  disagreement  with  the  statements,  but  the  extent  to 
which  you  agree/disagree.  Note  that  the  statements  can  be  positively  or  negatively 
stated.  The  scale  is  straight-forward  —  there  is  no  attempt  to  be  "tricky."  The  next  page 
is  the  answer  sheet . 

Answer  all  questions  based  on  your  present  experience  and  opinions  with  Continental 
aircraft.  For  the  purpose  of  these  questions,  consider  the  word  "automation"  to  mean 
autopilots,  autothrottles,  flight  directors  etc.  as  well  as  the  more  advanced  flight 
guidance  and  controls  that  you  are  familiar  with. 

1 .  Flying  today  is  more  challenging  than  ever. 

2.  I  am  concerned  that  automation  will  cause  me  to  lose  my  flying  skills. 

3.  Automation  leads  to  more  efficient,  safer  operations. 

4.  It  is  important  to  me  to  fly  the  most  modern  plane  in  my  company's  fleet. 

5.  As  I  look  at  aircraft  today,  I  think  they’ve  gone  too  far  with  automation. 

6.  Automated  cockpits  require  more  cross-checking  of  crew  members'  actions. 

7.  In  the  aircraft  I  am  presently  flying,  it  is  easy  for  the  captain  to  monitor  and 

supervise  the  first  officer 

8.  I  am  very  apprehensive  about  going  through  this  transition. 

9.  Automation  frees  me  of  much  of  the  routine,  mechanical  parts  of  flying  so  I  can 
concentrate  on  "managing"  the  flight. 

10.  lam  not  concerned  about  making  errors,  as  long  as  we  follow  procedures  and 
checklists. 

11.  I  look  forward  to  more  automation  -  the  more  the  better. 

12.  I  have  no  trouble  staying  "ahead  of  the  plane". 

13.  There  is  too  much  workload  below  10,000  feet  and  in  terminal  areas. 

1 4.  Automation  does  not  reduce  total  workload. 

1 5.  It  is  easy  to  bust  an  altitude  in  today's  environment. 

16.  lam  concerned  about  the  reliability  of  some  of  the  automation  equipment. 

17.  lam  concerned  about  the  lack  of  time  to  look  outside  the  cockpit  for  other  aircraft. 

18.  Continental's  CRM  training  has  been  helpful  to  me. 

19.  Some  times  I  feel  more  like  a  "button  pusher"  than  a  pilot. 

20.  I  regularly  maintain  flying  proficiency  by  turning  off  automation  and  hand  flying. 
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ATTITUDES-TOWARD-AUTOMATION  ANSWER  FORM 


Referring  to  the  20  statements,  place  an  "X"  in  the  box  that  best  represents  your  feeling 
about  the  statement.  Answer  quickly  --  your  first  impression  is  the  best  Be  sure  that 
you  respond  to  all  20  statements. 


strongly 

agree 

agree 

neither 
agree  nor 
disagree 

disagree 

strongly 

disagree 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
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NASA  CONTINENTAL  HUMAN  FACTORS  PROJECT 
QUESTIONNAIRE  NO.  2  Project  ID  Code  * 


1 

5 

*  If  you  have  forgotten  your  project  ID  code,  please  write  your  name  on 
the  top  of  the  page  -  we  will  remove  it  and  write  in  your  ID  code,  and  send  you  the 
code. 


This  is  the  second  in  the  series  of  NASA  questionnaires.  Please  fill  it  out  and  return  it 
to  us  in  the  enclosed  envelope.  You  should  receive  the  next  one  about  a  year  from 
now.  We  will  take  a  random  sample  of  our  volunteer  pilots  for  face-to-face  interviews  in 
Houston  and  may  see  you  then.  Again,  our  thanks  for  participating  in  the  study. 


Note  that  the  questionnaire  has  three  parts. 

1 .  Some  information  about  your  present  status  (on  this  page). 

2.  A  24-item  questionnaire  on  your  attitude  toward  automation. 

3.  Four  open-ended  questions  where  we  ask  you  to  give  us  some  answers  in  your 
own  words. 


Total  flying  time  in  B-757: _ hours.  Current  base: _ 

Following  your  training  and  IOE,  were  you  assigned  to  your  former  plane,  or  the  757? 
_ 757  Former  plane,  which  was _ 

If  it  was  former  plane,  how  long  was  it  before  you  returned  to  757?  _ months 

If  it  was  the  former  plane,  did  you  experience  any  problems  when  you  returned  to  the 
757? 

_ No  _ Yes  (please  describe) 
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OPEN-ENDED  QUESTIONS  -  Q2  AND  Q3 


Please  answer  the  following  questions.  If  you  need  more  space,  please  write  on  the 
back  of  the  page. 


1 .  What  did  you  think  of  your  training  for  the  757?  Did  you  have  trouble  with 
anything?  What  topics  should  receive  more  or  less  emphasis?  Please  comment 
on  the  training  aids  and  devices. 

2.  Describe  in  detail  an  error  which  you  have  made,  or  have  seen  someone  else 
make,  with  the  automation,  that  might  have  led  to  some  undesirable 
consequence.  How  could  it  have  been  avoided?  (equipment  design,  training, 
CRM,  procedures?) 

3.  What  can  you  say  about  crew  coordination  and  procedures  in  the  757?  In  what 
way  are  they  different  from  previous  planes  you  have  flown?  What  areas  can 
use  improvement? 

4.  How  would  you  compare  the  overall  workload  in  the  757  compared  to  your 
previous  plane?  Please  mention  anything  that  you  feel  should  be  changed  to 
help  you  manage  workload  (procedures,  ATC,  training,  etc?) 


The  first  four  questions  were  repeated  on  Questionnaire  3,  and  the  following  two  were 
added: 


5.  Please  tell  us  your  strategy  for  selecting  the  various  HSI  modes.  Do  you  always 
use  the  map  mode?  For  what  maneuvers,  if  at  all,  do  you  use  the  compass  rose 
mode?  The  expanded  VOR  or  I LS  mode? 

Can  you  also  give  us  an  estimate  of  the  per  cent  time  for  each? 

Map _  Expanded  ILS _  Expanded  VOR _ 

Rose _ 

Question  is  not  open-ended.  It  was  added  at  the  request  of  the  757  fleet  manager. 

6.  Note:  this  question  for  those  who  have  left  the  757. 

After  you  left  the  757  and  went  to  another  aircraft,  what  was  your  reaction?  What  did 
you  miss  about  the  757  avionics  and  automation?  What  did  you  like  better  about  the 
older  technology  plane?  Why  did  you  bid  off  of  the  757? 

Plane  and  seat  you  went  to:  Aircraft _  Seat _ 
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ADDITIONAL  ATTITUDE  PROBES  ON  QUESTIONNAIRES  2  AND  3 


Four  attitude  probes  that  were  not  appropriate  for  Q1  were  added  to  Q2  and  Q3.  The 
attitude  probes  on  Q2  and  Q3  were  identical. 

Additional  Probes: 


21 .  The  757  works  great  in  today's  ATC  environment. 

22.  Training  for  the  757  was  as  adequate  as  any  training  that  I  have  had 

23.  Electronic  flight  instruments  (“glass  cockpits”)  are  a  big  advance  for  flight  safety. 

24.  There  are  still  modes  and  features  of  the  757  automation  that  I  do  not 
understand. 
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NASA/CONTINENTAL  HUMAN  FACTORS  PROJECT 
QUESTIONNAIRE  NO.  3 


Project  ID  Code  * 

1 

5 

If  you  have  forgotten  your  ID  code,  please  write  your  name  on  the  top  of  this  page  - 
we  will  remove  it  and  write  in  your  ID  code,  and  send  you  the  code. 


This  is  the  third  and  final  in  the  series  of  NASA  questionnaires  in  this  project.  Please  fill 
it  out  and  return  it  to  us  in  the  enclosed  envelope.  Again,  our  thanks  for  participating  in 
the  study. 

Note  that  the  questionnaire  has  three  parts. 

I.  Some  information  about  your  present  status  (on  this  page). 

II.  A  24-item  questionnaire  on  your  attitude  toward  automation 

III.  Some  open-ended  questions  where  we  ask  you  to  give  us  some  answersinyour 
own  words.  There  is  a  special  question  for  those  of  you  who  have  left  the  75/  for 
another  aircraft. 


I. 

Current  aircraft  and  seat:  Aircraft _  Seat _ 

Total  flying  time  B-757: _ hours.  Current  base: _ 

Do  you  “feel  comfortable”  in  the  757  now?  Yes _  No _ 

If  yes,  how  long  after  you  went  on  the  line  did  it  take?  _ months: 


co\docs\quest\q1 23.app 
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Marginal  Homogeneity  Tests:  Q2  Vs.  Q3 


For  explanations,  see  page  C-0 
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Continental’s  Automation  Philosophy  Statement  (1994) 


The  purpose  of  enhanced  autoflight  and  flight  guidance  systems  in  our  current  and  future 
generation  of  aircraft  is  to  improve  precision  and  reduce  workload. 

The  pilot  must  be  well  versed  in  utilizing  the  full  capabilities  of  the  automated  systems  in 
the  aircraft.  Standard  operating  procedures  for  each  aircraft  have  been  developed  to 
establish  conventional  methods  for  utilizing  automated  functions  in  any  given  phase  of 
flight.  However,  it  is  ultimately  up  to  the  judgment  of  the  pilot  as  to  how  the  automation  is 
employed. 

If  an  automated  function  improves  precision  and/or  reduces  workload  in  a  given  situation, 
then  its  use  may  be  desirable.  If  an  automated  function  does  not  complement  the 
situation,  the  use  of  a  more  basic  mode  displays  good  judgment. 

Continental  Airlines  expects  its  pilots  to  match  the  level  of  automation  to  the  flight 
dynamics  of  the  moment.  Automated  functions  are  tools.  The  pilot’s  judgment  is  the 
master  of  those  tools.  If  automation  helps,  use  it.  If  it  hinders,  go  back  to  basics.  Match 
the  resource  to  the  situation. 
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AUTOMATION  COMMITTEE’S  RECOMMENDED  FORMAT 


The  Automation  Committee  recommended  a  more  graphic  format,  so  that  the  levels  stood 
out  in  their  relative  vertical  position  in  the  automation-to-manual  stack.  This  design  was 
never  adopted. 


Continental’s  goal  for  automation  is  to  increase  safety  and  efficiency,  and  improve 
situational  awareness,  while  reducing  pilot  workload.  Pilots  must  be  proficient  in  all 
capabilities  of  their  aircraft  including  the  automated  systems  and  must  use  their  judgment 
as  to  how  those  systems  are  employed.  Standard  Operating  Procedures  for  use  of 
automated  features  have  been  developed  for  each  of  Continental's  fleets  and  may  be 
found  in  respective  flight  manuals. 


LEVELS  OF  AUTOMATION 


HAND  FLOWN 


RAW  DATA 


II 


HAND  FLOWN 


FLIGHT  GUIDANCE 


III 


AUTOPILOT/ 

AUTOTHROTTLE 


FLIGHT  GUIDANCE 


IV  LNAV  /  VNAV 


FLIGHT  GUIDANCE 
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FLIGHT  OPERATIONS  MANUAL  VERSION  (rev.  1998) 


This  version  is  included  in  all  flight  operations  manuals  (FOMs): 


The  goal  of  automation  in  our  aircraft  centers  around  safety  and  efficiency.  The  purpose 
of  enhanced  autoflight  and  flight  guidance  systems  in  our  current  and  future  generation 
of  aircraft  is  to  improve  precision,  reduce  workload,  and  contribute  to  situational 
awareness. 

The  pilot  must  be  proficient  in  utilizing  all  capabilities  of  the  systems  in  the  aircraft  from  the 
most  basic  hand  flown  mode  to  the  full  use  of  the  aircraft’s  most  automated  function. 
Standard  operating  procedures  for  each  aircraft  have  been  developed  to  established 
conventional  methods  for  utilizing  automated  functions  in  any  given  phase  of  flight. 
However,  it  is  ultimately  up  to  the  judgment  of  the  pilot  as  to  how  automation  is  employed. 


Third  and  fourth  paragraphs  same  as  1994  version  (page  F-1 ). 
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APPENDIX  G 


Continental  Human-Centered  Automation  Training  (H-CAT) 


Simulator  Check  Airmen  CRM  Briefing  Outline  (PC/RT/LOFT) 


Standardized  Briefing  Guidelines 


The  following  outline  will  be  incorporated  into  all  simulator  PC’s,  RT’s,  and  LOFTS.  The 
intent  is  to: 

1 .  Inform  crews  of  the  recent  increase  in  incidents  that  have  occurred  in  the  past  12 
months  and  the  resulting  FAA  surveillance  program. 

2.  Heighten  the  emphasis  on  Crew  Resource  Management  throughout  the  airline. 

3.  Elevate  CRM  from  a  “nice  to  know”  to  a  “need  to  know”  status. 

4.  Ensure  that  pilots  are  routinely  exposed  to  a  baseline  of  standard  CRM  briefing 
items. 

5.  Introduce  CRM  elements  and  outline  the  role  they  play  in  all  facets  of  our  operation. 

6.  Enhance  the  uniformity  of  training,  i.e.  take  one  step  closer  to  seamless  training. 


It  is  extremely  important  to  include  the  entire  crew  in  the  CRM  brief.  Instructors  working 
with  three-man  crews  will  present  the  CRM  brief  before  splitting  up  to  discuss  particular 
PC  or  RT  maneuvers.  Active  participation  by  all  crew  members  is  critical  to  the  success 
of  the  program.  Utilization  of  facilitation  and  reverse  briefing  techniques  is  encouraged 
wherever  possible. 


Reference:  Enhancing  Flightdeck  Safety.  Continental  Airlines  Quarterly  Standards 
Meeting,  April  14,  1994. 
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APPENDIX  H 


ASRS  Reports  Concerning  Automation  and  Training 


1 .  At  approx  90  miles  south  of  SNS  on  J88  given  FL240.  Approx  50  miles  south  of 
SNS  given  clearance  to  cross  55  miles  south  of  SJC  VOR  at  or  below  1 7000',  25 
miles  south  of  SJC  VOR  at  and  maintain  8000'.  F/O  at  this  time  figure  new 
waypoints  to  make  these  restrictions  and  entered  them  into  CDU.  Aircraft  started  to 
descend  automatically  following  instructions  given  from  FMC.  Approx  SNS  VOR, 
OAK  Center  asked  what  our  altitude  was.  At  that  point  we  were  going  through 
22000,  way  above  our  clearance  of  at  or  below  17000'  55  south  of  SJC.  During  this 
narrative  with  ATC  the  aircraft  started  to  turn  left  and  manually  turned  back  toward 
GILRO  intersection.  Cause  of  altitude  incursion:  descend  waypoints  entered  into 
computer  at  wrong  spot  on  legs  page.  Circumstances:  Captain  checked  on  aircraft 
two  months  prior  to  flight.  No  flight  time  given  in  aircraft.  Deemed  qualified  in 
aircraft  by  FAA  standards.  First  flight  in  aircraft  since  checkout.  PC  check  in 
simulator  next  day.  F/O  had  approx  1000  hours  in  aircraft.  Was  not  aware  of  mistake 
in  waypoint  insertion  in  FMC  due  to  lack  of  experience  in  aircraft.  Conclusion:  FAA 
deems  pilot  qualified  on  aircraft  A  model  with  short  course  on  differences  between 
aircraft  B.  No  recent  experience  qualifications.  This  aircraft  is  not  the  same.  The 
aircraft  in  question  should  have  a  distinct  type  rating  for  a  pilot  to  be  considered 
qualified.  Along  with  this  the  currency  requirements.  At  present,  once  a  pilot 
completes  differences  training,  he  is  considered  qualified.  He  might  not  fly  the 
aircraft  for  a  year  or  more  but  still  be  deemed  qualified  by  the  FAA. 

Recommendation:  separate  type  rating  for  this  aircraft  and  the  associated  training  and 
currency  requirements.  Supplemental  information  from  Accession  71794.  We 
missed  the  crossing  restriction  by  5000'.  I  believe  we  began  the  descent  too  late  and 
to  complicate  the  situation  further,  the  computer  was  programmed  incorrectly  for  the 
crossing  restriction.  I  also  feel  an  experienced  Captain  would  have  caught  the 
discrepancy  between  the  VHF  navigation  DME  and  the  information  given  by  the 
computer.  (Accession  #71850) 


2.  Experienced  failure  of  one  flight  management  computer  (FMC)  and  had  requested 
clearance  toward  Boston  and  started  in  that  direction.  We  then  contacted  Dispatch  to 
find  out  if  they  wanted  the  aircraft  at  BOS  or  JFK  to  replace  the  failed  FMC.  We 
knew  that  if  one  FMC  failed  prior  to  the  oceanic  gateway,  we  had  to  land.  But 
Dispatch  had  another  answer,  and  that  was  that  the  flight  was  legal  to  continue.  They 
mentioned  another  flight  had  the  same  problem  a  few  nights  earlier,  and  our  event 
was  even  agreed  to  by  our  maintenance  people  in  Tulsa.  They  both  concurred  the 
MEL  requirement  for  2  FMC's  for  extended  range  (ER)  operations  was  for  dispatch 
purposes  only.  So  we  proceeded  across  to  Paris.  We  looked  in  the  operating  manual, 
Pilot  Operating  Handbook,  MEL,  and  could  not  find  any  requirement  to  land  if  one 
FMC  failed  prior  to  the  oceanic  gateway.  Upon  arrival  at  DFW  from  Paris  a  few  days 
later,  we  were  met  by  a  flight  manager  who  pointed  out  the  references  as  to  the  fact 
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that  we  should  have  landed.  But  they  were  not  in  the  abnormal  procedures  section,  as 
they  should  have  been,  but  buried  in  the  normal  section  where  we  did  not  look  for  a 
problem.  This  procedure  has  now  been  moved  to  the  proper  section,  and  the  MEL 
will  be  changed  to  reflect  this.  Dispatch  and  maintenance  have  been  corrected  on  this 
matter.  Contributing  to  this  incident  is  the  fact  that  the  international  ground  school 
can  lead  you  to  believe  that  the  aircraft  has  3  long  range  navigation  systems.  When  in 
fact  you  only  have  two.  Three  inertial  reference  systems  (IRS)  and  1  FMC  equals  1 
long  range  navigation  system.  Those  same  3  IRS’s  and  the  other  FMC  equals  the 
other  long  range  navigation  system.  Failure  of  either  FMC  brings  you  down  to  one 
long  range  navigation  system.  This  is  not  emphasized  in  the  ground  schools  as  we  are 
so  used  to  flying  with  3  INS,  or  Omega  systems.  I  am  going  to  recommend  that  this 
be  emphasized  in  the  schools.  (Accession  #75956) 


3.  Our  medium-large-transport  (non-EFIS)  was  cleared: '  MUSEC  4  departure,  TRM 
transition  at  or  below  3000  ft  until  6  DME,  maintain  4000  ',  expect  FL370  after  1 0 
minutes.'  We  set  up  the  cockpit  and  briefed  the  departure  in  accordance  with  our 
company  operations  manual  for  the  Santa  Ana  noise  abatement '  normal  cutback ' 
procedure.  Distractions  were  as  follows:  New  captain  (first  non-checkride  line 
departure  from  Santa  Ana),  new  first-officer  (first  line  trip  ever),  no  APU  (requiring 
airstart  at  gate,  and  decision  from  captain  to  perform  noise  abatement  procedure 
bleeds  on  (normally  it  is  bled  off,  with  APU  for  pressurization),  unfamiliar  aircraft 
(neither  of  us  had  much  time  in  the  non-EFIS  medium-large-transport),  unfamiliar 
clearance  (specifically  the  '  below  3  until  6  DME '  part).  Our  company  procedure 
calls  for  a  maximum  performance  takeoff,  flaps  15  degrees  (normally  reduced  thrust, 
flaps  5  ),  28  degree  nose  up  body  angle  to  1000  ft,  then  a  radical  thrust  reduction, 
simultaneous  flap  retraction  to  5  degrees,  and  a  shallow  climb  at  1  /  2+15  to  6  DME, 
on  autopilot  (vertical  speed  mode  +200  fpm,  engaged  after  thrust  cutback).  It  is  a 
challenging  procedure  even  with  practice,  but  for  2  new  guys,  no  APU,  new 
cockpit.. .it  starts  to  add  up.  Now  consider  the  very  heavy  GA  activity  at  and  around 
Santa  Ana  on  a  Sat,  and  you  start  to  get  the  picture  of  the  scene  in  this  flight  deck.  I 
made  the  takeoff  and  cutback  (to  77.6  %  N1  based  on  an  extremely  light  weight  of 
88000  lbs)  and  attempted  to  engage  the  autopilot  at  1000  ft  as  per  procedure.  Rate  of 
climb  at  this  point  was  well  in  excess  of  6000  fpm  (VSI  needle  pegged).  At  that 
moment,  we  were  handed  off  to  departure  by  ATC,  and  advised  to  watch  for  VFR 
traffic  1 2  o'clock  at  3000  ft  (typical).  I  lowered  the  nose  and  told  my  first-officer  to ' 
stay  outside  and  keep  your  eyes  open.'  The  autopilot  would  not  engage,  and  in  the  2  - 
3  secs  I  spent  trying  to  figure  it  out  (unfamiliar  cockpit),  the  altitude  hom  went  off.  I 
thought  it  was  the  first  alert  (2000  for  3000  ft),  but  in  the  initial  climb  we  both  missed 
that  one,  and  it  was  the  second  alert  (you  blew  it!).  We  were  at  3300  ft  and  our 
ballistic  path  carried  us  to  3500  ’  at  about  4.5  DME.  I  briskly  lowered  the  nose  and 
reduced  thrust.  ATC  asked,  'what  was  your  assigned  altitude?'  My  first-officer 
replied '  3000  ft..’  I  added,  'We  're  correcting.'  ATC  answered  crisply,  Roger.'  He 
then  cleared  us  to  13000  ft,  and  advised  tis  again  of  Traffic  12  o'clock,  3  miles, 
northbound  along  the  coast  at  3000  ft.  In  my  opinion  the  Santa  Ana  noise  abatement 
procedures  are  an  extreme  menace  to  aviation  safety  and  should  be  abandoned  at 
once.  This  flight  had  all  the  necessary  ingredients  for  disaster:  new  crew  (both 
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captain  and  first-officer),  new  aircraft  (in  the  fleet  for  quite  some  time  but  both  pilots 
relatively  unfamiliar),  radical,  one-of-a-kind,  maximum  performance,  totally 
nonstandard  departure  procedures  (well  practiced  in  simulator,  but  done  only  once 
before  by  captain  on  company  line  check),  heavy  GA  traffic,  extremely  busy  flight 
deck,  high  deck  angle  (28  degrees)  making  see  and  avoid  a  complete  farce,  unfamiliar 
departure  clearance.  I  strongly  recommend  the  following  actions:  a  thorough  review 
of  all  non-FAA  imposed  noise  abatement  regulations  and  procedures.  A  spotlight  on 
Santa  Ana  in  particular  (that  airport  is  an  accident  waiting  to  happen,  ground 
operations  ramp,  are  just  simply  crazy).  Scheduling  guidelines  that  preclude  the  new 
captain/new  first-officer  scenario  any  time  the  flight  is  into  severe  weather  or  into 
particularly  difficult  airports.  Standardization  of  airline  cockpits,  or  assignment  of 
flight  deck  crew  to  only  one  variant  (medium-large-transport  A  or  medium-large- 
transport  B  or  medium-large-transport  B  EFIS  only).  Advising  ATC  to  simplify 
departure  clearances  as  much  as  possible  at  all  times,  but  particularly  when  extra 
conditions  (weather  noise  abatement  rules,  etc.)  are  imposed.  (Accession  #99595) 


4.  We  were  cleared  to  cross  40  nautical  miles  west  of  LINDEN  VOR  to  maintain 
FL270.  The  captain  and  I  began  discussing  the  best  method  to  program  the  CDU  to 
allow  the  performance  management  system  to  descend  the  aircraft  We  had  a 
difference  of  opinion  on  how  to  best  accomplish  this  task  (since  we  are  trained  to  use 
all  possible  on-board  performance  systems).  We  wanted  to  use  the  aircraft's 
capabilities  to  its  fullest.  As  a  result,  a  late  descent  was  started  using  conventional 
autopilot  capabilities  (vertical  speed,  max  indicated  mach/airspeed  and  speed  brakes). 
Near  the  end  of  descent,  the  aircraft  was  descending  at  340  kias  and  6000'  fpm  rate  of 
descent.  The  aircraft  crossed  the  fix  approx  250-500'  high.  Unfortunately,  we  made 
no  call  to  ATC  to  advise  them  of  the  possibility  of  not  meeting  the  required 
altitude/fix.  This  possible  altitude  excursion  resulted  because:  1)  Captain  and  F/O 
had  differences  of  opinion  on  how  to  program  the  descent.  2)  Both  thought  their 
method  was  best:  the  captain's  of  programming  (fooling)  the  computer  to  believe  anti¬ 
ice  would  be  used  during  descent,  which  starts  the  descent  earlier;  the  F/O’s  of 
subtracting  5  miles  from  the  navigation  fix  and  programming  the  computer  to  cross  5 
miles  prior  to  LINDEN  at  FL270.  3)  A  minor  personality  clash  between  the  captain 
and  F/O  brought  about  by  differences  of  opinion  on  general  flying  duties,  techniques 
of  flying  and  checklist  discipline.  4)  Time  wasted  by  both  Captain  and  F/O 
(especially  F/O)  in  incorrectly  programming  CDU  and  FMS  for  descent,  which 
obviously  wasted  time  at  level  flight,  which  should  have  been  used  for  descent. 
Observation:  as  a  pilot  for  a  large  commercial  carrier  at  its  largest  base,  we  seldom  fly 
with  the  same  cockpit  crew  member.  This  normally  does  not  create  a  problem.  I  do, 
however,  feel  that  with  the  "new  generation"  glass  cockpits  being  on  the  property 
approx  6  years;  this  can  cause  a  bit  more  difficult  transition  than,  say  month  to  month 
cockpit  crew  change  on  a  727  or  pre-EFIS  DC-9.  I  have  flown  commercially  for  10 
years,  and  have  flown  2-man  crew  aircraft  for  8  of  those  1 0.  The  toughest  transition 
for  me  is  to  determine  who  shares  PF  and  PNF  duties.  This  historically  (3  years)  has 
been  most  difficult  when  the  other  crew  member  has  transferred  from  a  3-man 
cockpit  to  a  2-man  "glass  cockpit."  This  is  especially  pertinent  when  the  crew 
member  has  been  on  a  3-man  crew  aircraft  for  a  #  of  years.  As  F/O,  when  you  are  the 
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PNF,  you  accomplish  your  normal  duties.  However,  often  times  when  one  is  the  PF, 
he  also  has  to  do  the  PNF  duties  because  the  other  crew  member  has  not  been  used  to 
doing  PNF  duties  to  the  extent  that  it  is  required  on  2-man  cockpits,  whether  they  be 
conventional  or  EFIS.  This  obviously  can  lead  to  a  myriad  of  probs.  Add  weather 
problems  or  an  airport  such  as  Washington  National,  LGA  or  Orange  county,  and 
problems  can  accelerate  with  alarming  rapidity.  (Accession  #122778) 


5.  Situation:  failure  to  make  crossing  restriction  on  Arrival  route.  The  captain  was 
flying  and  I  was  handling  the  radios  and  FMC  work.  After  programming  the  ATC 
crossing  restriction  in  the  FMC  we  still  had  about  40  miles  before  reaching  the  fix. 

At  this  time,  I  told  the  captain  that  we  were  high  on  the  profile  and  he  acknowledged. 

I  then  began  to  prepared  our  landing  data  and  complete  the  required  company 
communications.  The  captain  took  over  ATC  until  1  could  complete  my  other  duties. 
At  some  point  during  this  process,  I  noticed  that  we  were  still  high  and  I  again  said 
"we  look  high  to  me"  and  repeated  the  crossing  restriction  to  the  captain.  He 
acknowledged  and  deployed  the  speed  brakes.  When  I  finished  the  leading  data  we 
had  about  5  miles  before  reaching  the  restriction  and  I  then  realized  there  was  no  way 
we  would  make  the  crossing  altitude.  1  told  the  captain  and  then  informed  ATC.  One 
factor  involved  was  the  captain  was  new  with  only  35  hours  on  the  narrow 
body/widebody.  I  have  over  a  year  on  the  widebody,  but  this  was  my  first  [narrow 
body]  trip.  While  the  narrow  body/widebody  are  considered  to  be  one  "type"  by  the 
FAA  and  my  company,  they  have  many  differences.  One  of  these  being  performance, 
so  when  I  thought  we  were  high,  we  were  really  very  high,  and  the  action  taken  by 
the  captain  to  correct  the  descent  path,  which  I  thought  would  be  adequate,  wasn't. 
Along  with  this,  the  training  I  received  for  the  [narrow  body]  was  inadequate. 

Ground  school  consisted  of  6  hours  of  [narrow  body]  differences  that  was  nothing 
more  than  "filling  the  squares"  for  the  FAA.  We  didn't  even  have  all  of  the  [narrow 
body]  revisions  for  our  widebody  manuals  at  the  time,  and  there  wasn't  any  required 
flight  or  simulator  training  for  the  first  officers.  Other  factors  include  my  distraction 
away  from  the  descent  profile  in  the  form  of  company  communication,  preparation  of 
the  landing  data  and  informing  the  cabin  crew  of  our  position  so  they  could  prepare 
the  cabin  for  landing.  At  the  time  1  thought  I  was  still  in  the  "loop"  enough  to  be 
aware  of  what  was  going  on,  but  obviously  I  wasn't.  Also  maybe  if  I  had  told  the 
captain  that  it  didn't  look  we  would  make  the  crossing  restriction  instead  of  saying  we 
looked  high  he  might  have  made  a  greater  effort  to  get  down.  (Note:  after  years  of 
operating  widebody  aircraft,  my  company  has  just  started  adding  the  [narrow  body]  to 
the  fleet).  (Accession  #124072) 

6.  Departed  SFO  on  runway  OIL  and  tracked  SFO  350  degree  right  per  SID.  Checked 
in  with  departure  control  per  Tower.  Departure  Control  cleared  us  to  2 1 0  degree 
heading  and  4000  feet.  F/O  (PNF)  misunderstood  heading  and  a  few  seconds 
attention  over  the  correct  heading  was  taken  by  both  pilots,  the  captain  (PF)  did  not 
hear  the  new  altitude  and  did  not  notice  F/O  reset  altitude  in  mode  control  panel  from 
previous  cleared  altitude  of  FL230.  The  aircraft  was  climbing  at  approx  4000  fpm 
and  by  the  time  altitude  deviation  was  noticed  by  F/O  an  altitude  of 4400  feet  was 
momentarily  attained  before  return  to  4000'.  2  factors,  I  believe,  underlie  this 
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deviation.  The  first  is  that  this  was  my  first  trip  in  the  aircraft  since  "shotgun".  I  am 
also  new  to  the  "glass  cockpit"  and  feel  that  a  certain  amount  of  my  attention  is 
occupied  in  interpreting  the  glass  cockpit  instrument  presentation,  in  this  respect,  I 
feel  that  having  only  had  4  simulator  rides  in  training  with  the  glass  cockpit  is 
insufficient,  although  it  is  FAA  approved.  This  does  not  develop  enough  proficiency 
to  apply  in  the  real  world  with  the  multiplicity  of  contingencies  that  can  and  do  occur. 
The  second  factor  is  that  just  prior  to  the  altitude  and  heading  clearance,  I  had  called 
for  an  infrequently  used  but  SOP  autoflight  function.  This  caused  the  F/O  to  stop  and 
consider  his  response  and  thereby  interrupt  his  "flow"  during  this  busy  time  of  aircraft 
configuration  cleanup,  frequency  changes  and  reception  of  communications.  My  last 
remaining  thought  is  that  I  should  be  more  decisive  during  my  preflight  and  brief  the 
F/O  more  thoroughly  on  what  to  expect  from  me  in  the  way  of  post  takeoff 
procedures.  (Accession  #125079) 

7.  While  climbing  after  takeoff  to  13,000'  we  "overshot"  the  assigned  altitude  by  500' 

(1 3,500)  and  immediately  leveled  back  to  1 3,000.  Related  factors:  both  pilots  type 
rated  [narrow  body]  widebody.  Both  pilots  initially  trained  on  and  experienced  on 
widebody.  [Narrow  body]  and  widebody  flown  interchangeably  by  same  crews.  In 
that  [narrow  body]  and  widebody  are  common  type  rating,  once  having  the  initial 
check  out  in  one  no  further  aircraft  checkout  required  for  the  other.  It’s  possible  for  a 
legal  [narrow  body]  crew  to  have  never  flown  the  aircraft  and  be  assigned  a  revenue 
flight!  This  was  my  4th  leg  in  the  [narrow  body].  This  was  the  captain's  1  st!  While 
it  is  extremely  common  [narrow  body]  to  the  widebody  there  are  subtle  differences 
that  are  distracting  if  it's  your  first  encounter  with  the  [narrow  body].  In  this  case  I 
missed  my  1 000'  before  level  off  call  because  I  was  distracted  by  either  being 
assigned  or  mis-selecting  the  appropriate  radio  frequency.  Keeping  in  mind  that  the 
light  [narrow  body]  is  climbing  at  500  fpm.  I'm  not  used  to  this.  The  captain  surely 
wasn't.  And  for  his  first  flight  in  the  airplane  to  be  climbing  at  400  fpm  in  clouds  and 
snow  with  copilot  that  is  also  new  to  the  aircraft,  is  only  stacking  the  deck  against 
yourself.  In  essence  I  was  "given"  the  job  of  "checking  out  the  new  guy"  but  I  don't 
have  enough  experience  in  the  [narrow  body]  to  do  this  and  watch  everything  else.  I 
would  most  strongly  urge  that  we  return  to  the  policy  of  sending  a  check  airman  with 
each  Captain  for  a  few  legs.  Let  the  new  guy  work  out  the  kinks  with  someone  on 
board  that  is  trained,  comfortable  and  familiar  with  watching  the  whole  operation  and 
the  other  pilot  should  the  need  arise.  (Accession  #129814) 

8.  At  cruise  altitude  captain  went  to  restroom.  Clearance  was  given  to  cross  a  fix  at 
19,000'  and  to  change  to  another  frequency.  The  frequency  and  18,000  were  read 
back.  At  this  time  the  captain  returned  to  the  cockpit.  The  changeover  freq.  was 
dialed  in  but  the  new  controller  was  not  contacted.  With  the  help  of  the  captain  the 
crossing  restriction  was  loaded  in  the  FMC  and  it  was  determined  that  speed  brakes 
were  needed  to  make  the  restriction.  On  level  off  at  1 8,000  we  were  told  to  return  to 
1 9,000  and  contact  the  controller  as  originally  instructed.  We  were  in  VFR 
conditions  and  were  informed  by  the  controller  that  no  other  aircraft  were  in  the  area 
at  1 8,000.  The  F/O  just  completed  IOE  on  the  [wide  body]  and  was  an  engineer  for  4 
1/2  years  prior  to  this.  I  feel  that  an  instructional  tape  describing  dos  and  don'ts  of  2- 
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man  crew  everyday  operation  might  have  some  assistance  for  a  crew  member 
transition  from  a  3-man  crew.  (Accession  #132083) 

9.  Check  airman  in  right  seat  of  glass  cockpit  aircraft,  instructing  on  Captain's  IOE,  was 
trying  to  maximize  instruction  time  in  all  phases  of  flight.  During  later  stages  of 
descent,  discussing  descent  in  VNAV  and  other  modes,  ATC  issued  a  clearance  to 
cross  SEAGO  waypoint  at  1 1 000  ft  and  250  knots.  Shortly  thereafter,  we  received  an 
off  course  vector  and,  after  another  brief  period,  a  vector  to  intercept  the  Arrivals 
route.  Check  airman,  as  PF,  did  not  arm  LNAV  for  the  intercept  as  it  was  close  to  90 
degrees,  which  would  result  in  a  rapid  course  capture  and  a  more  radical  turn  than  one 
might  wish.  The  plan  was  to  turn  to  intercept  using  heading  mode  and  capture  LNAV 
when  closer  to  track.  Further  discussion  about  the  aircraft  distracted  both  pilots  and 
they  flew  through  the  Arrivals  route.  The  vectors  had  also  interrupted  the  descent 
profile.  The  ATC  controller  called  to  ask  whether  we  intended  to  intercept  and  with 
this  'wake  up'  call  we  did  reverse  and  intercept,  abeam  SEAGO  descending  through 
13000  ft.  Instructing  in  the  niceties  of  the  aircraft  had  taken  our  attention  from  the 
business  of  flying  the  clearance.  After  landing  on  the  west  side  of  DFW,  we  were 
cleared  to  cross  the  inbound  parallel  runway,  cross  a  bridge  and  contact  ground 
control  on  ffeq.  xxxx.  We  crossed  the  runway,  crossed  the  bridge,  but  check  airman 
got  busy  again  and  didn't  call  ground  control  until  about  half  way  to  a  distant  gate. 
Fortunately,  this  was  a  low  traffic  period  and  there  were  no  apparent  conflicts.  The 
moral  is  obvious:  pay  attention  to  the  business  at  hand.  The  luxury  of  the  third  crew 
member  is  no  longer  available  in  recent  aircraft  (excepting  long  haul)  and  more 
diligence  is  required  of  both  pilots.  It  is  much  too  easy,  and  not  uncommon,  for 
instructors  to  become  involved  in  extolling  the  virtues  of  'gee  whiz'  airplanes  versus 
'no  whiz'  ones  and  not  devote  the  necessary  attention  to  precise  operation.  (Accession 
#184917) 

10.  Aircraft  was  discovered  off  course  to  the  north.  A  turn  south  was  made.  Shortly 
thereafter,  I  checked  the  'cross-track'  on  #2  INS  and  found  it  to  be  20  miles  left.  I 
would  estimate  total  off  course  error  was  25-30  miles.  Intending  course  at  the  time 
was  westbound  from  57  degrees  north  30  degrees  west  to  55  degrees  north  40  degrees 
west.  Autopilot  was  discovered  to  have  defaulted  from  'nav'  mode  to  'heading'  mode. 
Heading  bug  was  set  approx  30  degrees  right  of  intended  course.  Aircraft  was  plotted 
on  course  at  a  position  of  approx  56  degrees  40  min  north  32  degrees  00  west  about 
10  mins  prior  to  incident.  Autopilot  will  default  from  'nav'  to  'heading'  during  a 
course  transfer  on  EFIS  course/heading  panel,  but  this  function  wasn't  accomplished, 
so  I  have  no  idea  how  autopilot  got  to  heading  mode.  Normal  procedure  calls  for  one 
INS  to  remain  in  'course'  page  for  a  readout  of  track  error  (distance  from  track 
centerline)  I  was  conducting  IOE  training  at  the  time  and  going  over  various 
functions  of  both  INS  units;  catalogs,  data  pages,  etc.,  so  neither  INS  was  in  course 
page.  On  analog  type  aircraft  HSI  needle  would  be  full-scale  deflection  before  being 
10  miles  off  course.  On  EFIS  presentation  in  'map'  mode  and  600  miles  scale  20 
miles  off  course  is  hardly  noticeable.  My  flight  director  was  showing  a  command  to 
turn  left,  but  this  is  not  unusual  when  the  opposite  side  INS  is  controlling  the 
autopilot.  On  analog  aircraft  if  the  autopilot  defaults  from  'aux  nav'  (INS  controlling 
autopilot)  it  goes  to  'turn  nob'  --  which  maintains  current  heading.  On  EFIS  aircraft  a 
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default  goes  to  "heading'  which,  depending  on  bug  setting,  can  cause  a  rapid  departure 
from  intended  course,  as  happened  to  us.  I  believe  increasing  crew  awareness  to  the 
fact  that  an  'off-course'  situation  will  not  be  displayed  in  the  same  dramatic  fashion  on 
EFIS  aircraft  as  on  analog  type  displays  is  important.  I  also  believe  crews 
transitioning  to  EFIS  equip  should  be  aware  of  the  importance  of  scanning  ADI  nav 
mode  displays  which  show  what  is  controlling  the  autopilot.  My  own  inexperience 
on  EFIS  aircraft  (approx  100  hours)  contributed  to  this  incident.  (Accession 
#223697) 

1 1 . 1  was  on  the  first  leg  of  my  IOE  on  a  widebody.  (My  first  flight  of  the  aircraft.)  The 
flight  was  LAX/ATL.  As  we  began  the  Rome  Arrival,  ATC  asked  us  to  cross  50 
northwest  of  Rome  at  FL290,  descend  to  FL240.  I  tried  unsuccessfully  to  enter  the 
restriction  in  the  FMS.  After  3  attempts,  the  captain  tried  unsuccessfully  and  tried  to 
explain  why  it  wouldn't  take  it.  Meanwhile,  no  descent  was  started.  Captain  said  'just 
descend  manually,  I'm  going  for  the  ATIS.'  However  I  descended,  it  wasn't  fast 
enough,  especially  with  a  70  kt  tailwind.  As  we  neared  the  50  NW  point  from  Rome, 
the  controller  became  more  concerned  and  asked  if  we  understood  the  clearance,  and 
what  our  DME  was.  I  said  we  understood,  but  didn't  reply  regarding  to  DME  because 
I  wasn't  sure  what  he  meant,  from  where.  As  I  was  calculating  distances,  he  came 
back  and  asked  if  we  could  be  out  of  FL290  in  30  seconds.  We  were  just  under  FL 
310  at  the  time  and  I  said  yes.  We  made  it,  continuing  down  to  FL  240.  We  had  the 
conflict  aircraft  in  sight  for  about  43  seconds  to  a  minute,  as  we  went  by  1500  ft  - 
2000  ft  above  us  at  about  11-1  o'clock.  The  captain  was  back  in  the  loop  at  about 
the  '30  second'  request,  and  Captain  also  set  FL240  in  the  MCP,  as  that  was  missed 
when  the  clearance  was  first  received.  It  is  very  clear  to  me  what  the  problem  was 
here.  A  big  part  of  it  was  my  first  leg  flying  the  airplane,  but  also,  we  are  flying  an 
airplane,  not  a  computer.  My  focus  on  the  FMS  got  in  the  way  of  my  doing  a  very 
simple  aircraft  descent  profile.  I  will  be  focusing  on  flying  first,  programming 
second.  Supplemental  info  from  Accession  259900:  When  1  diverted  my  attention  to 
getting  the  ATIS  we  had  approx  30  miles  to  go  to  the  crossing  fix  and  to  lose  6000  ft 
this  was  not  a  problem.  However,  a  very  slow  descent  occurred  due  to  the  F/O’s 
unfamiliarity  with  the  descent  features  in  the  FMS.  One  problem  here  is  the  training 
in  ground  school  and  the  simulator  doesn't  always  give  a  pilot  a  good  knowledge  of 
descent  features  of  the  FMS  due  to  the  fact  that  most  of  the  simulator  work  is  pattern 
altitudes.  To  the  extent  possible,  this  check  airman  from  now  on  forward  will  not 
assume  that  the  new  pilot  will  yet  have  a  good  operating  knowledge  of  the  equipment 
and  when  those  times  that  require  diversion  of  attention  to  other  duties  to  keep  a 
closer  eye  on  the  aircraft.  (Accession  #259889) 

1 2.  This  was  my  first  trip  on  this  aircraft  without  training  people  aboard.  This  is  still  a 
brand  new  aircraft  and  none  of  us  pilots  have  had  much  exposure  or  experience  flying 
in  it.  We  were  on  the  CIVET  profile  descent  to  runway  25L  at  LAX.  Our  crossing 
restriction  was  14000'  to  CIVET.  We  misinterpreted  our  instruments  and  began 
descent  to  1 0000',  believing  we  were  inside  CIVET.  At  about  1 3000'  the  LAX 
approach  controller  told  us  that  we  started  down  early  and  needed  to  maintain  14000' 
until  CIVET.  We  immediately  climbed  to  14400',  the  assigned  altitude  to  CIVET. 
After  rechecking  our  instrument  we  realized  that  our  DME  reading  was  based  on 
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FUELER  intersection  instead  of  the  LAX  localizer  DME.  I  feel  this  was  an  easy 
mistake  to  make  considering  our  very  limited  exposure  to  this  aircraft.  I  find  the 
glass  cockpit  a  very  difficult  system  to  master  and  a  frightfully  easy  way  to  make 
critical  mistakes— at  least  when  the  pilot  is  new  to  it.  The  problem  occurred  when 
both  of  us  mistook  the  DME  for  FUELER  intersection  displayed  on  the  nav  display 
for  the  LAX  DME,  a  smaller  font  image  on  the  primary  flight  display.  The  fix  for 
this  problem,  I  believe,  is  more  training  for  the  crews.  Checkouts  have  become 
extremely  costly  forcing  airlines  to  make  them  in  the  shortest  time  possible,  which  is 
understandable.  However,  I  think  more  training  would  help  pilots  with  this  extremely 
complex  new  flight  system.  Callback  conversation  with  reporter  revealed  the 
following:  reporter  cites  that  this  was  first  trip  w/o  a  check  airman  on  board.  Also 
states  that  this  flight  crew  was  very  low  on  combined  experience  as  the  captain  had 
only  30  hours  of  experience.  That  is  counting  the  25  hours  obtained  on  IOE  time. 
Reporter  also  states  that  the  1 5  hours  he  had  as  operating  experience  was  3  takeoffs 
and  landing's  and  the  rest  of  the  time  was  logged  from  the  jump  seat.  Reporter  feels 
that  this  is  too  little  exposure  to  the  real  world  of  operating  a  $125,000,000  aircraft 
and  that  he  was  overworked  in  the  Arrival  and  got  confused  as  the  captain  started  the 
descent  prematurely.  He  was  of  no  assistance  in  preventing  the  deviation.  This  event 
occurred  in  spite  of  3  years  and  9  years  operating  time  on  standard  747's  for  F/O  and 
Captain.  It  could  be  suggested  that  if  at  all  possible,  2  low  time  pilots  should  not 
mixed  together  as  a  flight  crew.  The  coordinated  crew  concept  suffers  from  the 
composite  low  experience  level  and  exposes  the  aircraft,  crew,  and  passengers  to  an 
unnecessarily  high  risk  of  incident,  deviation,  or  accident.  The  economics  as 
practiced  in  this  low  training  hours  approach  cannot  be  justified  considering  the 
possible  results  from  a  mix  of  unfortunate  circumstances  being  thrown  to  a  set  of  low 
in  type  pilots  in  an  ever  changing  and  ever  increasingly  complex  environment. 
Providing  the  best  in  hands  on  experience  and  training  should  be  the  goal  and  it  is 
considered,  from  a  historical  viewpoint,  that  F/O's  should  obtain  their  operating 
experience  in  the  seat  that  they  would  normally  function  and  therefore  be  of  more 
assistance  to  the  PIC.  Jump  seat  riding  should  not  be  considered  for  operating 
experience  in  this  complex  aircraft.  Callback  conversation  #2  with  reporter  revealed 
the  following:  the  primary  flight  display  (PFD)  was  on  "ILAX"  showing  ILS/DME 
distance  from  25L  at  LAX.  Nav  display  (ND)  showed  mileage  to  waypoint  in  stored 
route.  Reporter  could  not  explain  why  mistake  was  made  when  all  the  waypoints 
were  in  the  stored  route  of  the  FMC.  The  FMC  system  auto  selects  the  required  radio 
for  nav  display  with,  in  this  case,  the  25L  "ILAX"  ILS/DME  being  selected.  The 
ILS/DME,  according  to  reporter  would  not  be  auto-selected  automatically  until  about 
30  DME  out  unless  "forced"  through  selection  and  activation  of  certain  push  buttons 
near  the  screen.  The  "time"  attached  to  CIVET  waypoint  was  not  considered  in  the 
election  for  descent.  The  "bottom  line"  in  the  assessment  of  this  event  is  training  and 
the  amount  of  technical  expertise  that  is  introduced  to  the  student  in  that  training 
atmosphere.  There  is  a  level  of  certainty  in  the  future  of  the  "glass  cockpit"  and  its 
portrayal  of  valuable,  usable  data.  This  however  comes  about  through  repeated  use 
and  experience.  Initial  training  that  disallows  hands  on  use  in  the  "formative  hours" 
can  only  be  previewing  another  altitude  deviation  or  misinterpretation  that  may  have 
more  serious  considerations.  The  potential  for  error  in  a  low  time  flight  crew  must  be 
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re-emphasized  as  an  evaluation  is  made  of  further  comments  from  reporter.  On  this 
aircraft  there  is  no  ACARS  system  thus  requiring  the  PNF,  in  addition  to  his  other 
duties  to  contact  the  company  with  landing  ETA  and  gate  info.  Add  to  this,  on  a 
"CIVET  STAR"  the  fact  that  LAX  airport  constantly  uses  the  task  inducing  procedure 
of  runway  switching  to  facilitate  aircraft  departures.  Consider  the  additional 
workload  to  re-program  the  FMC  by  getting  into  the  pages  of  the  CDU  and  selecting 
the  newly  assigned  runway/ILS  for  approach.  Proper  crew  coordination  would  then 
dictate  another  task  induced  approach  plate  review.  (Accession  #307372) 
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